NASA Contractor Report 166616 


C£-| (,UI4 P-U 


N AS A-CR- 16661 6-PT-2 
19860003886 


Mathematical Analysis Study for 
Radar Data Processing and 
Enhancement 

Part II: Modeling of Propagation 
Path Errors 

Robert James and James D. Brownlow 


Contract NAS2-11491 
August 1985 


Lsiia - •• ; '» i 


NOV 2 5 1985 

LANGLEY RESEARCH CENTER 
LIBRARY, NASA 
.NAMPJQN, VIRGINIA 


NASA 

National Aeronautics and 
Space Administration 



NF02169 






NASA Contractor Report 166616 


Mathematical Analysis Study for 
Radar Data Processing and 
Enhancement 

Part II: Modeling of Propagation 
Path Errors 


Robert James and James D. Brownlow 
GMD Systems, Lancaster, California 


Prepared for 
Ames Research Center 
Dryden Flight Research Facility 
Edwards, California 
under Contract NAS2-11491 

1985 


NASA 

National Aeronautics and 
Space Administration 
Ames Research Center 
Dryden Flight Research Facility 
Edwards, California 93523 


SV 




TABLE OF CONTENTS 


1.0 INTRODUCTION 1-1 

1.1 Definition of Terms 1-1 

2.0 DESCRIPTION OF REFRACTION METHODS ANALYZED 2-1 

2.1 Gradient Refraction Correction Method 2-1 

2.2 The Spherical Slab Refraction Correction Method 2-1 

2.3 The Cyber Refraction Correction Method 2-2 

3.0 MODELING METHODS USED IN THIS ANALYSIS 3-1 

4.0 PHASE I - COMPARISON WITH JSC RESULTS 4-1 

4.1 Comparison of Solutions for Nsl of 0.0002550 4-2 

4.2 Comparison of Solutions for Nsl of 0.0003307 4-5 

4.3 Comparison of Solutions for Nsl of 0.0003860 4-8 

4.4 Results of Phase I Analyses 4-11 

5.0 PHASE II - COMPARISONS USING EDWARDS ATMOSPHERES .... 5-1 

5.1 Edwards EHA-75 Hot Day 5-3 

5.2 Edwards ECA-75 Cold Day 5-7 

5.3 Edwards Cold Moist Morning 5-11 

5.4 Edwards Warm Moist Morning 5-15 

5.5 Edwards Warm Day Afternoon 5-19 

5.6 Results of Phase II Analysis 5-23 

6.0 REFERENCES 6-1 

APPENDIX A - TABULATED DATA NSL = 0.0002550 A-l 

APPENDIX B - TABULATED DATA NSL = 0.0003307 B-l 

APPENDIX C - TABULATED DATA NSL = 0.0003860 C-l 

APPENDIX D - TABULATED DATA FOR EHA-75 HOT DAY D-l 

APPENDIX E - TABULATED DATA FOR ECA-75 COLD DAY E-l 

APPENDIX F - TABULATED DATA FOR COLD MOIST MORNING F-l 

APPENDIX G - TABULATED DATA FOR WARM MOIST MORNING G-l 

APPENDIX H - TABULATED DATA FOR WARM DAY AFTERNOON H-l 

i-1 



MATHEMATICAL ANALYSIS STUDY 
FOR 

RADAR DATA PROCESSING AND ENHANCEMENT 


PART II - MODELING OF PROPAGATION PATH ERRORS 


1.0 INTRODUCTION 

In accordance with requirements set forth in contract 
NAS2-1149, propagation path errors were modeled using three 
refraction correction methods. 

1. The gradient refraction method (A900 computer) 

2. The spherical slab correction method (Varian 73 computer) 

3. The White Sands (Pearson) method (Cyber 7028 computer) 

Each of these methods is described briefly below. For 
detailed mathematical algorithms used by each, refer to refer- 
ence 8 for the gradient refraction solution, to reference 9 
for the spherical slab solution, and to reference 10 for the 
Cyber solution. 

Solutions from each of these methods were obtained for ranges 
out to 150 or more nautical miles and for altitudes up to 
100,000 feet. Results were compared with tabulated data from 
high-precision refraction correction algorithms described in 
reference 9. Also analyzed were propagation path effects 
using five reference atmospheres selected to cover the 
extremes of weather conditions at Edwards. 

1.1 Definition of terms 

The term 'exact' is used in reference to correction methods 
which use iterative processes in which the accuracy is 
limited only by the speed and precision of the computer, and, 
of course, by the knowledge of the atmospheric composition. 

If the latter is known, then with sufficient time and 
computer precision, any specified degree of accuracy can be 
obtained. These solutions are literally an integration 
process in which delta-t is taken closer and closer to zero to 
obtain an ever more accurate solution. 

The term 'approximate' is applied to empirically-determined 
methods which attempt to obtain quick solutions that are 
reasonably close to the results from the 'exact' methods or 
to experimental data that has been accumulated as a result of 
long term observations. The term 'error' is applied to the 
difference between the tested solution and the 'exact' 
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solution being used as a reference, or to the difference 
between the tested solution and experimental data. 

It should be noted that the results of 'exact' solutions are 
only as good as the estimates of the atmospheric composition, 
and, therefore, may be far from exact on any given day when 
the atmosphere is not well behaved. Therefore, the accuracy 
of any of the tested refraction techniques must be accepted 
with that limitation. 

The symbol 'n' refers to index of refraction (n = 1 for a 
vacuum). The symbol 'N' refers to refraction modulus, 
refactivity, or modulus of refraction all of which equal 
n - 1. The term N-units applies to the value N times 10 to the 
6th power. For example a refractivity of 0.0003307 would be 
330.7 N-units. 
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2.0 


DESCRIPTION OF REFRACTION METHODS ANALYZED 


2.1 The Gradient Refraction Correction Method 

The gradient refraction method falls into the category of an 
'exact' solution. In the gradient method, two rays are 
incrementally projected outward from the source, starting at 
the measured elevation angle. The amount of bending in each 
incremental segment is determined from the refractivity 
gradient computed perpendicular to the direction of the ray 
travel. This causes a velocity differential between the 
upper and lower rays so that one ray (usually the upper) will 
travel slightly farther in the same time increment. This 
determines the amount of rotation in the wave front. At each 
new iteration, the rays are projected perpendicular to the 
angle of the new wave front as computed during the preceeding 
iteration. At each iteration step, the range remaining is 
decreased by the average refraction corrected path length of 
the ray within the cell. The values for refractivity and the 
refractivity gradient at the midpoint position of each 
segment or cell can be computed from a mathematical relation 
(such as when using exponential refractivity) or from a 
mathematical curve fit to real-world data. 

Gradient refraction was developed by GMD Systems and was 
reported in reference 8. It is believed to be superior to 
the conventional Snell's law approach since it works 
essentially with small angles, thereby eliminating register 
saturation conditions that accompany most Snell's law 
solutions, and it can achieve accuracies equivalent to other 
'exact' solutions using fewer iterations and without the need 
for a computer with an exceptional word length. Furthermore, 
the algorithms were designed to yield the maximum possible 
accuracy at very low (or even negative) angles. 

2.2 The Spherical Slab Refraction Correction Method 

The spherical slab solution method is an 'approximate' 
solution. It computes range and angle corrections based on 
surface refractivity using an exponential lapse rate. Of the 
several approximation methods in common use, the spherical 
slab solution was selected for use by JSC early in the Space 
Shuttle program because it provided slightly better results 
than other 'approximate' solutions tested. 

With the spherical slab method, a slab height, H*, is 
computed for each solution. Everywhere below the slab, the 
refractivity is the same as the surface value. Everywhere 
above the slab, the refractivity is zero (no refraction). 
Based on the measured range, Rm, and the measured elevation 
angle, Em, a number of empirical computations are performed 
to determine the slab height such that a ray projected from 
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the source at angle Em will travel in a straight line until 
reaching the surface of the slab. At that point, a single 
Snell's Law refraction will bend the ray so that it passes 
approximately through the target point. Needless to say, the 
key to this method is in the determination of the proper slab 
height which will cause this to occur for a particular 
combination of Rm and Em values. Reference 9 evaluated 13 
empirical relations for calculation of the elevation 
correction, and 5 for the calcuation of the range correction. 
The selected elevation correction is claimed to have a 
maximum error of 24.4 percent at 0 degree elevation, and 
negligible error above about 10 degrees. 

2.3 The Cyber Refraction Correction Method 

The third method was developed at White Sands Missile Range 
in 1958. It was intended to provide a rapid data reduction 
process that would eliminate the need for lengthy processing 
of atmospheric profile data. It's design goals were to 
provide results which would 'hold' for elevation angles from 
1 degree to 90 degrees over a range of 500 to 200,000 yards. 
The solution assumes an exponential lapse rate from the 
surface to an altitude where the refractive index is essen- 
tially constant over the entire year (about 24,000 feet MSL 
at White Sands) . The elevation correction uses two constants 
Kle and K2e. Kle is computed from the value of surface 
ref ractivity . Using the computed value of Kle, a 
corresponding value of K2e is obtained from a table of 
precomputed constants by using linear interpolation methods. 
The value of the elevation angle correction is then deter- 
mined by the relation: 

Ec = (Kle Rm cos Em)/(K2e Rm sin Em) 

A range correction of the same form is provided in reference 
10, however no range correction is used in the Cyber program. 
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3.0 


MODELING METHODS USED IN THIS ANALYSIS 


The modeling of results from each of the three methods was 
accomplished in two phases. The first phase compared the 
solutions from each of the three methods with tabulated 
results from an 'exact' solution described in reference 9. 

This solution was based on a JSC ray-tracing technique in 
which each result was generated in double precision (28 
decimal digits) on a CDC Cyber 74 computer. Results provided 
in the tables in reference 9 were based on 50,000 iterations 
for Em of 0 degrees, 5000 iterations for values of Em from 
0.5 to 2.5 degrees, and 500 iterations for values of Em of 3 
degrees or greater. The JSC ray-tracing solutions uses a 
conventional Snell's Law approach which subdivides the 
atmosphere into concentric shells. Refractivity within each 
shell is a constant value. Bending occurs as the ray crosses 
each shell boundary. Additional accuracy is obtained by 
increasing the number of shells, and hence the number of 
iterations, used for each solution. Obviously, refractivity 
values within the shell structure can be based on a 
mathematical relation between refractivity and altitude, or 
on real world values quantitized to the altitude intervals of 
the shells. Probably the greatest disadvantage of this 
method is that it works with large angles and requires a 
large word length to prevent register saturation. 

The tables provided in reference 9 were compiled specifically 
for the purpose of determining the accuracy of newly proposed 
'approximate' refraction correction methods, and were based 
on the modulus of refraction being an exponential function of 
altitude. Note that the modulus of refraction in the real 
world is never exactly exponential; however, the purpose of 
the JSC document was not to define the nature of the 
atmosphere but to determine how well individual refraction 
methods would yield accurate results with a completely 
defined, ideal exponential atmosphere. The document also 
does not attempt to compute the modulus of refraction for any 
specific psychrometric conditions, but provides results for 
three separate values of refractivity. In short, if the sea- 
level refractivity for the r-f energy is the same as the value 
given in any one table, and, if the atmosphere were ideally 
exponential in nature, then the results from other solution 
methods should match the 'exact' values given in the tables. 

In the second phase, two of the methods were compared against 
a third to determine how much difference would result when 
approximate solutions (using only surface refractivity 
values) were compared with a high-accuracy solution which 
used different vertical profiles, representative of both 
typical and extreme atmospheric compositions. The 
atmospheric compositions examined were: 
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1. EHA-75 hot day atmosphere 

2. ECA-75 cold day atmosphere 

3. A typical cold moist morning atmosphere 

4. A typical warm moist morning atmosphere 

5. A typical hot day afternoon atmosphere 

Although items 1 and 2 above do not include the amount of 
water vapor present at the various altitudes, estimates of 
these values were supplied by NASA in terms of dew point 
temperatures to accompany the temperature and pressure 
profiles. 
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4.0 


PHASE I - COMPARISON WITH JSC RESULTS 


During Phase I of the modeling of the three refraction 
methods, the measured range and measured elevation values 
from the JSC tables were used as inputs to all three solution 
methods. The JSC tables are structured such that measured 
range values are provided for specific measured elevation 
values of 0, 0.5, 1, 1.5, 2 , 2.5, 3, 3.5, 4, 4.5, 5, 6, 7, 8, 
9, 10, 12, 14, 16, 18, 20, 25, 30, 35, 40, 45, 50, 60, 70, 

80, and 90 degrees. Separate tables are provided for target 
true altitudes of 1000, 2000, 5000, 10,000, 20,000, 50,000, 
and 100,000 meters for three separate refractivity values: 
0.0002550, 0.0003307, and 0.0003860. This analysis called 
for ranges out to 150 nautical miles and altitudes up to 
100,000 feet. Therefore, tabulated values are provided in 
appendix A to appendix C. Appendix A contains values for 
target altitudes of 1000, 2000, 5000, 10,000, 20,000, 50,000, 
and 100,000 meters obtained using a sea level modulus of 
refraction of 0.000255; appendix B provides values for the 
same target altitudes using a modulus of refraction of 
0.0003307, and appendix C provides the same data for a 
modulus of refraction of 0.000386. In these tables, the 
target horizontal range goes from 0 out to about 640 nautical 
miles. 

The headings on each of the tables show the sea level modulus 
of refraction, the scale height, and the target altitude. 

The first two columns in the tables are the input data, the 
measured elevation and range values given in the JSC tables. 
This is followed by the corrected elevation and range values 
as computed by the gradient solution, the corrected elevation 
values computed by the Cyber (White Sands) solution, and the 
corrected elevation and range values computed by the Varian 
(spherical slab) solution. The next item in the table is the 
geometric altitude which would be obtained with no 
refraction, followed by the altitudes as computed from the 
gradient. Cyber, and spherical slab solutions. Note that the 
Cyber solution contains no range correction. 

Since it is difficult to assimilate all of the data contained 
in these tables, data plots were made for target altitudes of 
2000 meters (6562 feet), 5000 meters (16,404 feet), 10,000 
meters (32,808 feet), and 50,000 meters (164,042 feet). 

These are provided for each of the three values of surface 
refractivity used in Phase I of the modeling analysis. 
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4.1 


Comparison of Solutions for Nsl of 0.0002550. 


Plots for a surface refractivity (Nsl) of 0.0002550 are 
provided in figures 4-1 (a) to 4-1 (d). The gradient solution, 
shown as a solid line, yields altitude values whose maximum 
'error 1 is on the order of 0.1 percent below 0.5 degree; drop- 
ping to about 0.01 percent by 5 degrees. The altitude 
'error' in the spherical slab solution method, shown by the 
short dashed line, has a maximum value of about 5.6 percent 
for angles under 1 degree; dropping to under 1.0 percent 
around 2 degrees elevation and to about 0.05 percent by 5 
degrees elevation. The Cyber method, according to reference 
10, is limited to elevation angles above 1 degree. 

Discounting errors in angles below 1 degree, the maximum 
'error' in the Cyber solution is on the order of 0.45 percent 
decreasing to about 0.1 percent by 4 to 6 degrees elevation 
angle, depending on the target altitude. Although 'errors' 
for elevation angles above 30 degrees are not shown on the 
plots, all converge to the correct altitude except for the 
Cyber solution which has a small error remaining at 90 
degrees elevation angle because no range correction is made. 

In terms of actual altitude 'errors' the gradient solution has 
its maximum 'error' of 27 meters (88.6 feet) in the derivation 
of altitude for the 50,000 meter (164,041 foot) target being 
tracked at 0 degree elevation angle. The spherical slab 
'error' for the same tracking condition is 1,752 meters 
(5,748 feet). By the time the target has reached 1 degree 
elevation angle the 'error' in the gradient solution has 
dropped to 2 meters (6.6 feet), the 'error' in the spherical 
slab solution has dropped to 499 meters (1637 feet) , and the 
'error' in the Cyber solution is 769 meters (2522 feet). At 
a 5 degree elevation angle, the three 'error' values are 0 
meters, 50 meters (164 feet) , and 56 meters (184 feet) 
respectively. 

For a target at 10,000 meters (32,808 feet) altitude, the 
'errors' in the three solution methods are: for 0 degree 

elevation angle, 8 meters (26.2 feet) for the gradient 
solution and 607 meters (1991 feet) for the spherical slab 
solution; for 3 degrees elevation, 1 meter (3.28 feet) for 
the gradient solution, 11 meters (36 feet) for the spherical 
slab solution, and 28 meters (92 feet) for the Cyber 
solution; and for 10 degrees elevation, 0 meter for the 
gradient solution, 1 meter (3.28 feet) for the spherical slab 
solution, and 5 meters (16.4 feet) for the Cyber solution. 

Tabulated data for this test condition are provided in 
appendix A. 
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RETRACTION COMPARISON PLOT 
NSL - O.00O2550; TRRGET ALTITUDE - £000 METERS 
GMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VRRIAN (SHORT DASH) 



MEASURED ELEVATION ANGLE, DEGREES 


Figure 4-1 (a). Plots for target at 2000 meters altitude. 


REFRACTION COMPARISON PLOT 
N5L - 0.0002550; TRRGET ALTITUDE - 5000 METERS 
GMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VRRIAN (SHORT DASH) 



Figure 4-1 (b) . Plots for target at 5000 meters altitude. 
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REFRACTION COMPARISON PLOT 
NSL - 0,0002550; TARGET ALTITUDE - 10,000 METERS 
GMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VRRIfiN (SHORT DASH) 



MEASURED ELEVATION ANGLE, DEGREES 


Figure 4-1 (c) . 


Plots for target at 10,000 meters altitude* 


REFRACTION COMPARISON PLOT 
NSL - 0.0002550; TARGET ALTITUDE - 50,000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VAR I AN (SHORT DASH) 



Figure 4-1 (d). Plots for target at 50,000 meters altitude* 
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4.2 


Comparison of Solutions for Nsl of 0.0003307. 


Figures 4-2 (a) to 4-2 (d) show the same plots for a sea level 
refractivity of 0.0003307 and a scale height of 6631.54 
meters (21757.0 feet). This value is probably the closest to 
a typical Edwards day. The 'errors' in all three methods 
seem to be slightly greater with the increased refractivity. 
For the gradient solution, the error for the 0 degree eleva- 
tion track of a target at 50,000 meters (164,041 feet) is 46 
meters (150 feet) or about 0.1 percent. By 0.5 degree 
elevation, the 'error' in the gradient solution has decreased 
to about 2.5 meters (8 feet) or less. Above 3 degrees 
elevation, the 'error' is below 0.003 percent for all 
altitudes. 

For the same refractivity value, the spherical slab solution 
yields altitude results that have an 'error' of 1799 meters 
(5,902.2 feet) for a target at 50,000 meters (164,041 feet) 
being tracked at a 0 degree elevation angle. Also note that 
the spherical slab results are highly erratic at the lower 
elevation angles for all altitudes. For the same 50,000- 
meter target, the errors in the spherical slab solution 
decrease to a minimum at around 3 to 3.5 degrees and then 
again increase, hitting a second peak about 4 to 6 degrees. 

For a target at 2000 meters (6562 feet), the secondary peak 
yields an 'error' of only 1 meter (3.28 feet), however, for a 
target at 50,000 meters (164,041 feet) the secondary peak 
causes an altitude 'error' of 58 meters (190 feet). Above 10 
degrees elevation angle, the errors in the spherical slab 
solution drop below 0.01 percent at all altitudes. 

For the same refractivity conditions the Cyber solution yields 
'errors' that are equivalent to 1464 meters (4803 feet) for 
the 50,000 meter (164,041 foot) target being tracked at a 1 
degree elevation angle. The 'error' on the same high 
altitude target drops to 50 meters (164 feet) by the time the 
elevation angle has increased to about 3 degrees, then slowly 
diminishes to about 2 meters (6.5 feet) by the time the 
target is overhead. 

Tabulated data for this test condition are provided in 
appendix B. 
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REFRACTION C0MPRRI50N PLOT 
N5L - O.00O3307; TARGET RLTITUDE - £000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); V BRIAN (SHORT DRSH • 



Figure 4-2 (a). Plots for target at 2000 meters altitude 


REFRRCTION COMPARISON PLOT 
N5L - 0.0003307; TARGET RLTITUDE - 5000 METERS 



MEASURED ELEVATION ANGLE, DECREES 

Figure 4-2 (b). Plots for target at 5000 meters altitude 
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REFRRCTION COMPARISON PLOT 
NS l - 0.0003307; TRRGET ALTITUDE * 10,000 METERS 
CMD GRADIENT SOLUTION (SOLID)j CYBER (LONG BASH); VRR3 AN (SHORT DRSH) 



Figure 4-2(c). Plots for target at 10,000 meters altitude. 


REFRRCTION COMPRRISON PLOT 
NSL - 0.0003307; TARGET ALTITUDE - 50,000 METERS 
GMD GRADIENT SOLUTION (SOLID) j CYBER (LONG DRSH); VRRIRN (SHORT DRSH) 



0 5 10 15 20 25 30 

MEASURED ELEVATION ANGLE, DEGREES 


Figure 4-2(d). Plots for target at 50,000 meters altitude. 
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4.3 Comparison of Solutions for Nsl of 0.0003860. 

Values for the third JSC test condition with a sea level 
refractivity of 0.000386 are shown in figures 4-3 (a) to 4-3 (d) . 
Note that the Cyber data is not included in this plot, 
because the sea-level refractivity value exceeds the table 
limits used in the Cyber program. Again, the 'errors' in 
each method increase somewhat as the refractivity value 
increases. For the 50,000 meter target being tracked at a 
0-degree elevation angle, the gradient solution shows an 
'error' of 68 meters (223 feet) while the spherical slab 
solution shows an 'error' of about 1510 meters (4954 feet). 

For a target at 10,000 meters (32,808 feet) altitude, the 
maximum 'error' with the gradient solution is 23 meters (75 
feet) at 0 degrees elevation angle. For the same target and 
the same elevation angle, the spherical slab solution shows 
an 'error' of about 842 meters (2762 feet). By the time the 
elevation angle reaches 3 degrees, both solutions yield 
'errors' on the order of 1 meter (3.28 feet). 

As in the other cases, the spherical slab results are very 
erratic when the elevation angle is low. 

Tabulated data for this test condition are provided in 
appendix C. 
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RETRACTION COMPARISON PLOT 
NSL - . 0003060; TARGET ALTITUDE - 2000 METERS 
CMD CRRDIENT SOLUTION (SOLID); VfiRIfiN (SHORT DASH) 



Figure 4-3 (a). Plots for target at 2000 meters altitude. 


RETRACTION COMPARISON PLOT 
NSL - .0003960; TARGET ALTITUDE - 5000 METERS 
CMD CRRDIENT SOLUTION (SOLID); VfiRIfiN (SHORT DfiSH) 



Figure 4-3 (b) . Plots for target at 5000 meters altitude. 
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COMPUTED ALTITUDE, METERS ™ COMPUTED ALTITUDE, METERS 


REPRRCTION COMPRRI50N PLOT 
NSL - . O0Q366O: TARGET RLTITUDE - 10,000 METERS 
CMD GRADIENT SOLUTION (SOLID); VflRIflN (SHORT DRSH) 



MEASURED ELEVATION ANGLE, DEGREES 


4-3 (c). Plots for target at 10,000 meters altitude. 


REPRRCTION C0MPRRI50N PLOT 
NSL - .0003060; TRRGET ALTITUDE - 50,000 METERS 
CMD GRRDIENT SOLUTION (SOLID); VAR I AN (SHORT DASH) 



MEASURED ELEVATION ANGLE, DEGREES 

Figure 4-3 (d) . Plots for target at 50,000 meters altitude. 
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4.4 


Results of the Phase I Analyses 


The analyses of all three refraction methods currently in use 
at Dryden indicates that, when compared with data derived 
from an 'exact' method used for comparison purposes by JSC, 
the gradient solution used on the A900 computer at the radar 
site yields the best agreement. On a typical target at 
10,000 meters altitude, the maximum 'error' of 8 meters (26 
feet) occurs when the target is at a range of 211 nautical 
miles and the elevation angle is 0 degrees. By the time the 
target range has decreased to 178 nautical miles 
(0.5 degree elevation angle), the 'error' in determining the 
target altitude is only 2 meters (6.5 feet), and by a range 
of 62 miles, it has dropped to less then 1 meter (3.28 feet). 
The 'errors' in the spherical slab solution for the same 
target positions are 607 meters (1991 feet) , 238 meters (781 
feet), and 9 meters (29.5 feet), and the 'error' in the Cyber 
results (given only for the last point) is 17 meters (55 
feet) . By the time the target is in to a range of about 28 
miles, both the gradient solution and the spherical slab 
solution show 'errors' of less than 1 meter, while the Cyber 
solution still has an 'error' of approximately 4.5 meters (15 
feet) . This indicates that the gradient solution has about 
100 times the 'accuracy' of the spherical slab solution at 
elevation angles below 1 degree, but that the advantage 
decreases until, at about 10 degrees, the results from the 
two solutions are almost in agreement. The Cyber solution is 
invalid for elevation angles below 1 degree, but between 1 
and 2 degrees seems to have better accuracy than the 
spherical slab solution. Above 2 degrees the spherical slab 
solution seems to have a slight advantage over the Cyber 
method. 

Thus, it can be concluded that, for very low-elevation angle 
tracking requirements, only the gradient solution yields 
reasonably valid results. From the appearances of the plot 
for a target at 10,000 meters (32,808 feet), both the Cyber 
and spherical slab solutions must be considered to be invalid 
below about 2 degrees elevation angle. 

One additional comment should to be made regarding the 
'errors' based on the JSC 'exact' solution. The JSC solution 
used 500 iterations for elevation angles of 3 degrees and 
above, 5000 iterations for elevation angles of 0.5 to 2.5 
degrees, and 50,000 iterations for 0 degrees elevation angle. 
Note that on the 10,000 meter plot, the gradient refraction 
difference plot has a noticeable but small jump from 3.0 to 
2.5 degrees and a significant jump from 0.5 to 0 degrees. 
Since there was no change in the computational method used in 
the gradient solution at these points, the jumps can only be 
attributed to changes in the JSC results which occurred at 0 
degrees and in the range from 0.5 to 2.5 degrees where the 
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jumps are noted. Therefore’, it is entirely possible that, in 
spite of the large word length used in the JSC solution, 
roundoff error may nevertheless have caused inaccuracy in 
very low elevation values shown in the JSC tables. 
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5.0 Phase II - COMPARISONS USING EDWARDS ATMOSPHERES 

For this phase of the propagation path modeling, five Edwards 
atmospheres were used to determine the effects of non- 
exponential refractivity lapse rates on refraction 
corrections from the spherical slab and Cyber methods. 

In the Phase I analysis, all of the solution methods used 
surface refractivity values in order to determine the amount 
of correction to be applied to the measured target position 
values. With both the spherical slab solution and the Cyber 
solution, the exponential lapse rates are inherent in the 
solution methods. In the gradient solution, the refractivity 
values for 1000-foot altitude intervals were computed from 
the surface values using the exponental decay factor given in 
the JSC tables. For interim altitudes, an exponential inter- 
polation was used. 

In the phase II analysis, the psychrometric data profiles for 
each of the test atmospheres were used to compute the true 
refractivity for even 1000 foot altitude points required by 
the gradient refraction solution, and exponential 
extrapolations were used for the interim altitudes. Again 
the altitude comparisons were made for the altitude range 
from 1000 meters (3281 feet) up to 100,000 meters (328,084 
feet) . 

For this part of the analysis, the gradient refraction 
solution served as the 'exact' method. In order to arrive at 
the input values of measured range and elevation, a ray was 
propagated from the source starting at the selected elevation 
angle. The iteration was continued until arriving at the 
segment where the desired altitude was contained within the 
iteration cell. At this point, the final incremental range 
and angle changes were obtained by interpolation. The 
measured range was determined by the sum total of the 
refraction adjusted incremental range measurements. Measured 
range values for each increment were obtained from the 
relation 


Rmi = Rj_/ni, 

where Rmi is the incremental segment of measured range, Ri is 
the true incremental range, and ni is the index of refraction 
derived for the cell from the selected atmospheric profile. 
This value of Rm, along with the starting value of Em, served 
as the input parameters for the other two refraction correc- 
tion methods analyzed. 

The tabulated results from the Phase II analysis are provided 
in appendices D through H. As in the Phase I analysis, each 
table contains values for targets at altitudes of 1000, 2000, 
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5000, 10,000, 20,000, 50,000, and 100,000 meters. The 
results in appendix D were computed for the atmospheric 
conditions given for the EHA-75 atmosphere, and appendix E 
provides results for the ECA-75 atmosphere. Appendices F 
through H provide results for a typical cold moist morning, a 
warm moist morning, and a hot day afternoon. 

In all of these tables, the heading data gives the name of 
the test profile being analyzed, the computed sea level 
ref ractivity, and the scale height used by the spherical slab 
solution. The input data are the elevation angle and range 
determined from the gradient solution. The next columns 
provide the corrected elevation and range for the gradient 
and spherical slab solutions and the corrected elevation for- 
the Cyber solution. 

In the altitude section of the listing, the first column 
shows the geometric altitude which would be obtained without 
any refraction correction, the second column shows the values 
obtained with the gradient solution serving as the 'exact' 
method, and the third and fourth columns show the altitudes 
determined from the Cyber and spherical slab solutions 
respectively. A row of asterisks in the Cyber columns 
indicate that the refractivity value is outside the range of 
values in the Cyber program. 
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5.1 


Edwards EHA-75 Hot Day 


Psychrometric data for the EHA-75 atmosphere are provided on 
pages D-2 and D-3. 

Plots of the results of altitude calculations from the three 
solution methods for the psychrometric parameters associated 
with the EHA-75 atmosphere are provided in figures 5-1 (a) to 
5-1 (d) which represent altitudes of 2000, 5000, 10,000, and 
50,000 meters (6562, 16,404, 32,808, and 164,042 feet). 

For the 2000-meter (6562-foot) altitude, the spherical slab 
results, shown by the short dash, come into agreement with 
the gradient solution at about 3 degrees elevation angle. 

The Cyber solution comes into good agreement by about 2 
degrees. Although both the Varian and Cyber solutions have 
significant errors at the very low elevation angles, it 
appears that the Varian solution is more erratic at the lower 
elevation angles. 

On the 5000-meter (16,404-foot) plot, both the spherical slab 
and Cyber solutions come essentially into agreement with the 
gradient solution by about 8 degrees elevation angle. 

Results from both solutions have less than 10 meters (32.8 
feet) of 'error' from about 2 degrees upward and less than 4 
meters (13.1 feet) of 'error' from about 5 degrees upward. 

The Cyber solution does have a residual error of about one 
meter (3.28 feet) at the higher angles. This is because no 
range correction is made. 

The 10,000-meter (32,808-foot) altitude plot shows acceptable 
agreement above about 10 degrees elevation angle. At 
5 degrees elevation, both the Cyber and the Varian method 
have 'errors' of about 20 meters (65.6 feet). At 10 degrees 
elevation angle, the 'error' in the Varian solution is about 
2.5 meters (8.2 feet) and the 'error' in^the Cyber solution 
is about 7 meters (23.0 feet). Again the larger error in the 
Cyber solution can be attributed to the fact that no range 
corrections are made. 

The 50,000-meter (164,042-foot) altitude plot shows good 
agreement between the gradient and Cyber methods for 
elevation angles above about 20 degrees. The 'error' in both 
the Cyber and Varian methods is about 160 meters (525.0 feet) 
at 5 degrees elevation angle. Both solutions converge toward 
the gradient results as the elevation angles increase. In 
this case, the lack of range correction in the Cyber method 
causes a bias of about 10 meters (32.8' feet) in high-angle 
solutions. 

Again, the Cyber algorithms provided a smoother plot at the 
lower elevation angles where the Varian solution was erratic. 
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Table I provides a comparison of the 'errors' in the Cyber and 
Varian solutions. The first columns, labelled 'EXP,' show 
the errors inherent in the solution methods, even if the 
atmosphere were truly exponential in nature. The second 
columns, labelled ' EHA, ' show the 'error' in results from the 
the two ’solution methods when the atmosphere follows the 
EHA-75 composition. For this comparison, the exponential 
atmosphere having the 0.0003307 sea level ref ractivity value 
was used. Note that for targets at normal tracking altitudes 
of 10,000 meters (32,808 feet) and below and at elevation 
angles above 10 degrees, the errors are relatively small. 

For higher altitude targets or lower elevation angles, the 
errors due to the atmospheric variations can significantly 
increase the errors inherent in the solution method. 

More complete comparisons can be made using the tabulated data 
contained in appendix D. 


TABLE I - COMPARISON OF ERRORS (EHA-75) 


Altitude, 

El angle. 

Cyber 

error , 

Varian 

error , 

meters 

degrees 

meters alt. 

meters 

alt. 



EXP 

EHA 

EXP 

EHA 

2,000 

2 

-1 

0 

+2 

+2 


5 

0 

0 

0 

0 


10 

0 

0 

0 

0 

5,000 

2 

+2 

+10 

+9 

+13 


5 

+3 

+4 

+3 

+4 


10 

+2 

+2 

0 

0 

10,000 

2 

+11 

+44 

+39 

+70 


5 

+12 

+20 

+13 

+19 


10 

+2 

+5 

0 

+1 

50,000 

2 

-310 

+25 

+28 

+404 


5 

+41 

+159 

+58 

+171 


10 

+27 

+66 

+4 

+43 


20 

+10 

+19 

0 

+10 
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REFRACTION COMPRRISON PLOT 
EHR-75 HOT DRY; TARGET ALTITUDE - 2000 METERS 
GMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VARIAN (SHORT DRSH) 



MEASURED ELEVATION ANGLE, DECREES 


Figure 5-1 (a). Plots for target at 2000 meters altitude. 


REFRACTION COMPRRISON PLOT 
EHR-75 HOT DAY; TARGET ALTITUDE - 5000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VARIAN (SHORT DRSH) 



MEASURED ELEVATION ANGLE, DECREES 

Figure 5-1 (b) . Plots for target at 5000 meters altitude. 
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COMPUTED ALTITUDE, METERS COMPUTED ALTITUDE, METERS 


RETRACTION COMPARISON PLOT 
EHA-7S HOT DAY; TARGET RLTITUDE - 10.000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VARJAN (SHORT DASH) 



MEASUREO ELEVATION ANGLE, DECREES 


-1(c). Plots for target at 10,000 meters altitude. 


REXRRCTION C0MPRRI50N PLOT 
E HA-75 HOT DRY; TARGET RLTITUDE - 50,000 METERS 
CMD GRADIENT SOLUTION (SOLID) j CYBER (LONG DASH); VAR1AN (SHORT DASH) 



MEASURED ELEVATION ANCLE, DECREES 

Figure 5-1 (d) . Plots for target at 50,000 meters altitude. 
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5.2 


Edwards ECA-75 Cold Day 


Psychrometric data for the ECA-75 atmosphere are provided on 
pages E-2 and E-3. 

Plots showing the results of comparisons between the three 
refraction correction methods are provided in figures 5-2 (a) 
to 5-2 (d) . 

The 2000-meter (6562-foot) altitude plot for this atmospheric 
model shows excellent results from both the Cyber and Varian 
solutions for all elevation angles above about 2 degrees. 
However, the results degrade rapidly as the elevation angle 
decreases below 2 degrees. 

The 5000-meter (16,404-foot) plot shows 'errors' in the two 
solution methods to be under 10 meters (32.8 feet) for all 
elevation angles above 2.5 degrees, decreasing to under 5 
meters (16.4 feet) for all elevation angles above 4 degrees. 
The error in the Varian solution becomes negligible at 
elevation angles above 7 degrees. The Cyber solution still 
shows an error of 3 meters (9.8 feet) at 7 degrees elevation 
angle. Again, this error is due to the absence of a range 
correction in the Cyber solution. 

The 10,000-meter (32,808-foot) plot shows 'errors' of 21 to 
22 meters (68.9 to 72.2 feet) for both the Cyber and Varian 
solutions at 5 degrees elevation angle. By 10 degrees 
elevation angle, the Cyber 'error' has dropped to 7 meters 
(23.0 feet) and the Varian 'error' has dropped to 3 meters 
(9.8 feet). As the elevation angle increases above about 12 
degrees the elevation error in both solution methods is 
negligible; however, the range bias still remains in the 
Cyber solution and this results in an error of several meters 
in the altitude calculations. 

The 50,000-meter (164,042-foot) altitude plot is almost 
identical with the same plot for the EHA-75 atmosphere. At 
5 degrees, the 'errors' in the Cyber and Varian solutions are 
149 and 166 meters (488.8 and 544.6 feet), respectively. By 
10 degrees elevation, the 'errors' in the two solutions have 
decreased to 63 and 40 meters (206.7 and 131.2 feet), and by 
20 degrees elevation they have decreased to 18 and 9 meters 
(59.1 and 29.5 feet). 

Table II shows inherent errors in both solution methods when 
compared with the results from an exponential atmosphere 
(0.0003307 Nsl) in the real-world (gradient) solution and 
with the ECA-75 parameters in the real-world solution. Again 
the exponential 'errors' are found in the column labelled 
'EXP,' and the ECA-75 'errors' are found in the column 
labelled 'ECA.' 
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TABLE II - COMPARISON OF ERRORS (ECA-75) 


Altitude, 

El angle. 

Cyber 

error , 

Var ian 

error , 

meters 

degrees 

meters alt. 

meters 

alt. 



EXP 

ECA 

EXP 

ECA 

2,000 

2 

-1 

-1 

+2 

+2 


5 

0 

0 

0 

0 


10 

0 

0 

0 

0 

5,000 

2 

+2 

+9 

+ 9 

+16 


5 

+3 

+4 

+3 

+4 


10 

+2 

+2 

0 

0 

10,000 

2 

+11 

+45 

+39 

+76 


5 

+12 

+21 

+13 

+22 


10 

+2 

+7 

0 

+3 

50,000 

2 

-310 

-23 

+28 

+342 


5 

+41 

+149 

+58 

+166 


10 

+27 

+63 

+4 

+40 


20 

+10 

+18 

0 

+9 


Tabulated data for this test condition are provided in 
appendix E. 
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REFRACTION COMPARISON PLOT 
ECFW5 COLD DAY; TARGET ALTITUDE - 2000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VAR I AN (SHORT DASH) 



0 5 ID 15 20 25 30 

MEASURED ELEVATION ANCLE. DEGREES 


Figure 5-2 (a). Plots for target at 2000 meters altitude. 


REFRRCTION COMPARISON PLOT 
ECA-75 COLD DAY; TARGET ALTITUDE - 5000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VRRlflN (SHORT DASH) 



0 5 10 15 20 25 3D 


MEASURED ELEVATION ANGLE, DEGREES 

Figure 5-2 (b). Plots for target at 5000 meters altitude. 
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REFRACTION COMPARISON PLOT 
ECR-75 COLD DPT ; TARGET ALTITUDE - 10,000 METERS 



4 v — — 

MEASURED ELEVRTION ANGLE, DEGREES 

Figure 5-2(c). Plots for target at 10,000 meters altitude, 


REFRACTION COMPARISON PLOT 
ECR-75 COLD DRY; TARGET ALTITUDE - 50,000 METERS 
CMD CRRDIENT SOLUTION (SOLID); CYBER (LONG DR5H) j VRRIfiN (SHORT DRSH) 



MEASURED ELEVATION ANGLE, DEGREES 

Figure 5-2(d). Plots for target at 50,000 meters altitude. 
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5.3 


Edwards Cold Moist Morning 


The psychrometric data for the cold moist morning atmospheric 
model are provided on pages F-2 and F-3. 

Results for each solution method for altitudes of 2000 meters 
(6562 feet) , 5000 meters (16,404 feet), 10,000 meters (32,808 
feet), and 50,000 meters (164,042 feet) are provided in 
figures 5-3 (a) to 5-3 (d), respectively. 

Results from the 2000-meter (6562-foot) altitude plot 
indicate that both the Cyber and the Varian solutions yield 
excellent results for all elevation angles above about 2 
degrees. Results from both solutions below 2 degrees 
elevation angle are unusable. 

Data from the 5000-meter (16,404-foot) altitude plot show 
good agreement above 2.5 degrees elevation angle for both 
methods. Below 2.5 degrees the results are unusable. 

The 10,000-meter (32,808-foot) altitude plot shows that the 
Varian results become useable above about 8 degrees ('error' 
less than 4 meters (13.1 feet). Cyber results become useable 
at about 15 degrees. Below 2.5 degrees elevation angle, both 
solution methods fail. At 2.5 degrees elevation angle, the 
'errors' in both solutions are about 30 meters (98.4 feet). 

The 50,000-meter (164,042-foot) altitude plot again shows 
about the same behavior as the 50,000-meter plots for the 
previous test conditions. Significant 'errors' are present 
for all elevation angles above about 15 degrees. 

In all cases, the lack of range refraction correction in the 
Cyber solution causes unnecessary inaccuracies in the 
results, even at the higher elevation angles where the 
elevation correction is accurate. 

Table III provides a comparisons of 'errors' in the two 
solution methods based on gradient results with (1) an 
exponential atmosphere and (2) the cold moist morning 
atmosphere. The 'errors' between the results of the two 
solutions and the exponential (Nsl=0 .0003307) case are in the 
column labelled 'EXP.' The 'errors' between the results of 
the two solutions and the cold-moist-morning case are found 
in the columns labelled 'CMST. ' More detailed analyses can 
be made using the tabulated data in appendix F. 
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TABLE III - COMPARISON OF ERRORS (COLD MOIST MORNING) 


Altitude, 

El angle. 

Cyber 

error , 

Varian 

error , 

meters 

degrees 

meters alt. 

meters 

alt. 



EXP 

CMST 

EXP 

CMST 

2,000 

2 

-1 

-2 

+2 

+1 


5 

0 

0 

0 

0 


10 

0 

0 

0 

0 

5,000 

2 

+2 

+1 

+9 

+8 


5 

+3 

+2 

+3 

+3 


10 

+2 

+1 

0 

0 

10,000 

2 

+11 

+25 

+39 

+55 


5 

+12 

+17 

+13 

+17 


10 

+2 

+6 

0 

+2 

50,000 

2 

-310 

-102 

+28 

+253 


5 

+41 

+128 

+58 

+146 


10 

+27 

+57 

+4 

+34 


20 

+10 

+16 

0 

+7 
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REFRACTION C0MPRRI50N PLOT 


COLD MOIST MORNING; 7RRGET ALTITUDE - 2000 METERS 
CMfl GRADIENT SOLUTION (SOLID): CYBER (LONG DASH); VRR1RN (SHORT DRSH) 



MEASURED ELEVATION ANGLE, DECREES 


Figure 5-3 (a). Plots for target at 2000 meters altitude. 


REFRRCTION C0MPRRI50N PLOT 
COLD MOIST MORNING; TARGET ALTITUDE - 5000 METERS 



0 3 10 15 20 25 30 

MEASURED ELEVATION ANGLE, DECREES 


Figure 5-3 (b). Plots for target at 5000 meters altitude. 
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COMPUTED ALTITUDE, METERS COMPUTED ALTITUDE, METERS 


REFRACTION COMPARISON PLOT 
COLD MOIST MORNING; TARGET ALTITUDE - 10,000 METER 5 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VAR I AN (SHORT DRbH) 



MEASURED ELEVATION ANCLE, DECREES 


5-3 (c) . Plots for target at 10,000 meters altitude. 


REFRRCTION COMPARISON PLOT 
COLD MOIST MORNING; TARGET ALTITUDE - 50,000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VAR I AN (SHORT DASH) 



MEASURED ELEVATION ANCLE, DEGREES 

Figure 5-3 (d) . Plots for target at 50,000 meters altitude. 
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5.4 


Edwards Warm Moist Morning 


The psychrometric data for the warm moist morning atmospheric 
model are provided in table G-I (pp. G-2 and G-3). 

Results for each solution method for altitudes of 2000 meters 
(6562 feet), 5000 meters (16,404 feet), 10,000 meters (32,808 
feet), and 50,000 meters (164,042 feet) are provided in 
figures 5-4 (a) to 5-4 (d). Note that, the surface refractivity 
values for this test case fall outside the table limits for 
the Cyber solution, therefore only the gradient and spherical 
slab results are shown in the plots. 

Results from the 2000-meter (6562-foot) altitude plot 
indicate that the spherical slab solution comes into fair 
agreement with the gradient results at angles above about 2 
degrees, and essentially converges at elevation angles above 
about 4 degrees. 

Data from the 5000-meter (16,404-foot) altitude plot show 
useable data from the spherical slab solution at angles above 
about 4.5 degrees, and essentially identical results with the 
gradient solution at angles above 10 degrees. 

The 10,000-meter (32,808-foot) altitude plot shows that the 
spherical slab results are useable above about 4.5 degrees 
and provide good agreement with the 'exact' solution above 5 
degrees. 

The 50,000-meter (164,042-foot) altitude plot shows sizeable 
errors in the spherical slab results until above about 15 
degrees elevation angle. From about 2.5 to 15 degrees the 
'error' ranges up to about 145 meters (475.7 feet). 

Table IV provides a comparison of the 'errors' from the use 
of the Edwards warm moist morning with those present when the 
assumption is made that the real-world atmosphere is 
exponential. No Cyber data were available for this 
comparison. 
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TABLE IV - COMPARISON OF ERRORS (WARM MOIST MORNING) 


Altitude, 

El angle, 

Cyber 

error , 

Varian 

error. 

meters 

degrees 

meters alt. 

meters 

alt. 



EXP 

WMST 

EXP 

WMST 

2,000 

2 

-1 

**** 

+2 

-3 


5 

0 

**** 

0 

-1 


10 

0 

**** 

0 

0 

5,000 

2 

+2 

**** 

+ 9 

-26 


5 

+3 

*** * 

+3 

-4 


10 

+2 

**** 

0 

-2 

10,000 

2 

+11 

**** 

+39 

-35 


5 

+12 

**** 

+13 

-1 


10 

+2 

**** 

0 

-3 

50,000 

2 

-310 

** ** 

+28 

-28 


5 

+41 

* * * * 

+58 

+145 


10 

+27 

**** 

+4 

+39 


20 

+10 

**** 

0 

+9 


Tabulated data for this test condition are provided in 
appendix G. 
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REFRRCTION COMPARISON PLOT 
HARM MOIST MORNING; TRRGET ALTITUDE - £000 METERS 



Figure 5-4 (a). Plots for target at 2000 meters altitude. 


REFRRCTION COMPARISON PLOT 
WARM MOIST MORNING; TRRGET ALTITUDE - 5000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH) ; VAR I AN (SHORT DASH) 



0 3 10 15 £0 £S 30 

MEASURED ELEVATION ANCLE, DECREES 


Figure 5-4 (b) . Plots for target at 5000 meters altitude. 
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refrrction comparison plot 



Figure 5-4(c) . Plots for target at 10,000 meters altitude. 


REFRRCTION COMPARISON PLOT 
WARM MOIST MORNING; TARGET ALTITUDE - 50,000 METERS 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DRSH); VAR I AN (SHORT DASH) 



MEASURED ELEVATION ANGLE, DEGREES 

Figure 5-4(d). Plots for target at 50,000 meters altitude. 
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5.5 


Edwards Warm Day Afternoon 


The psychrometric data for the warm day afternoon atmospheric 
model are provided in table H-I, pages H-2 and H-3. 

Results for each solution method for altitudes of 2000 meters 
(6562 feet), 5000 meters (16,404 feet), 10,000 meters (32,808 
feet), and 50,000 meters (164,042 feet) are provided in 
figures 5-5 (a) to 5-5 (d) respectively. 

Results from the 2000-meter (6562-foot) altitude plot 
indicate that the results of both the Cyber and Varian 
solutions are valid above about 1.5 degrees. The results are 
invalid below that point. 

Data from the 5000-meter (16,404-foot) altitude plot show 
valid results with errors of 9 meters (29.5 feet) or below 
for elevation angles above 2 degrees. By 10 degrees, the 
Varian solution matches the gradient solution. At the same 
elevation angle, the Cyber method computes the correct 
elevation angle, however the inherent range bias causes a 
small error to remain in the altitude calculations. 

The 10,000-meter (32,808-foot) altitude plot shows that both 
the Cyber and Varian solutions have sizeable errors remaining 
for elevation angles of 5 degrees. By 10 or 12 degrees, the 
Varian solution obtains good agreement with the gradient 
method, but the Cyber solution still has an error of about 5 
meters due to the lack of range correction. 

The 50,000-meter (164,042-foot) altitude plot again shows 
agreement in the results from the Cyber and Varian solutions 
above 5 degrees, but both have 'errors' of about 170 meters 
(557.7 feet) at that point. By 10 degrees elevation angle, 
errors from about 45 to 70 meters (147.6 to 229.7 feet) are 
still present. At about 21 degrees elevation angle, the 
'errors' in the Varian solution finally go below 10 meters 
(32.8 feet). The 'errors' in the Cyber solution are almost 
double those in the Varian solution at the same point because 
of the range bias. 

Table V provides a comparison of the 'errors' when using 
atmospheric data from the Edwards warm day afternoon with 
those present when using the exponential real-world 
atmosphere. Again, the inherent use of an exponential 
atmosphere by both the Cyber and Varian refraction correction 
methods only causes minor errors for angles above 5 degrees 
for target altitudes at or below 10,000 meters (32808 feet). 
However, for the 50,000-meter target, the assumption of an 
exponential lapse rate can cause sizeable errors. 
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TABLE V - COMPARISON OF ERRORS (WARM DAY AFTERNOON) 


Altitude, 

El angle. 

Cyber 

error. 

Varian 

error , 

meters 

degrees 

meters alt. 

meters 

alt. 

• 


EXP 

WMAFT 

EXP 

WMAFT 

2,000 

2 

-1 

0 

+2 

+1 


5 

0 

0 

0 

0 


10 

0 

0 

0 

0 

5,000 

2 

+2 

+9 

+9 

+10 


5 

+3 

+4 

+3 

+3 


10 

+2 

+2 

0 

0 

10,000 

2 

+11 

+43 

+39 

+66 


5 

+12 

+19 

+13 

+18 


10 

+2 

+7 

0 

+3 

50,000 

2 

-310 

+57 

+28 

+433 


5 

+41 

+169 

+58 

+178 


10 

+27 

+69 

+4 

+46 


20 

+10 

+20 

0 

+10 


Tabulated data for this test condition are provided in 
appendix H. 
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REFRACTION COMPARISON PLOT 
WARM DA Y AFTERNOON; TARGET ALTITUDE - 2000 METERS 
CMD GRADIENT SOLUTION (SOLID) j CYBER (LONG DASH); VAR I AN (SHORT DA*H) 



MEASURED ELEVATION ANGLE, DEGREES 

Figure 5-5 (a) . Plots for target at 2000 meters altitude. 


REFRRCTION COMPARISON PLOT 
WARM DAY AFTERNOON; TRRGET ALTITUDE - 5000 METERS 


CMD GRADIENT SOLUTION (SOLID) j CYBER (LONG DASH); VRR3AN (SHORT DASH) 



MEASURED ELEVATION ANGLE, DECREES 


Figure 5-5 (b). Plots for target at 5000 meters altitude. 
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REFRACTION COMPARISON PLOT 
WARM DRY AFTERNOON; TRRGET ALTITUDE - 10,000 METER5 
CMD GRADIENT SOLUTION (SOLID); CYBER (LONG DASH); VRR1AN (SHORT DA-H) 



MEASURED ELEVATION ANGLE, DEGREES 


Figure 5-5(c). Plots for target at 10,000 meters altitude* 


REFRRCTION COMPARISON PLOT 
WARN DAY AFTERNOON; TARGET ALTITUDE - 50,000 METERS 
CMD GRADIENT SOLUTION (SOLID) j CYBER (LONG DASH); VAR I AN (SHORT DASH) 



MERSLIREO ELEVATION RNGLE, DEGREES 

Figure 5-5(d). Plots for target at 50,000 meters altitude. 
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5.6 


Results of the Phase II Analysis 


The phase II analysis provides a good comparison of the 
results of the three solution methods when non-exponential 
atmospheric conditions exist. Both the Cyber and Varian 
solutions appear to 'hold' down to the same elevation angles 
for targets at all altitudes. However, the Cyber solution 
has a slightly greater error noticeable at the higher 
elevation angles simply because there is no correction for 
the effects of refraction on the range measurements. If the 
Cyber solution is to be used in the future, then the simple 
range correction provided in reference 10 should also be 
incorporated. 

In both the phase I analysis and the phase II analysis, it 
was found possible to exceed the range of values in the Cyber 
refraction tables. This is shown as asterisks on the 
tabulated plots in the appendices. If this were to happen 
during an actual reduction of data on the Cyber, then an 
error message would print out advising the operator that the 
table values had been exceeded. Although the Cyber tables 
could easily be expanded to cover a wider range of 
atmospheric conditions and altitudes, it nevertheless should 
be noted that the Cyber method was devised to support missile 
testing at White Sands and was, according to information 
contained in reference 10, only intended for use at ranges 
from 500 to 200,000 yards (about 100 nautical miles). In 
this analysis, the Cyber routines did fail when the ranges 
were very long and the elevation angles very low, indicating 
that the published range limitations are reasonably correct 
for normal tracking missions. However, it was also found 
that the Varian (spherical slab) solution had approximately 
the same limits and yielded highly erratic results when the 
elevation values fell below those limits. 

With the addition of a range correction and with some 
expansion of the Cyber tables, the Cyber refraction 
correction method could be acceptable for normal aircraft 
tracking missions so long as the elevation angles were above 
values at which error limits for the specific mission were 
not exceeded. These can be easily determined from the 
tabulated data for the atmospheric conditions which most 
closely match the flight day conditions. In fact, if a range 
correction were incorporated, the Cyber solution would 
provide more consistent data as the elevation angle 
approached the failure point. Useable results, often with 
sizeable errors, can be obtained from both solutions down to 
the failure points. Below the failure points, the results 
are totally unusable. 

Failure points for both methods are approximately 1.5 degrees 
elevation angle for a target at 2000 meters (6,562 feet). 
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2 to 2.5 degrees for a target at 5000 meters (16,404 feet), 
2.5 to 3.5 degrees for a target at 10,000 meters, and about 
4 degrees for a target at 50,000 meters. Data obtained at 
elevation angles below the failure points will have extremely 
large errors, and data obtained just slightly above the 
failure points can still have sizeable bias errors. 

Both the Cyber and Varian refraction correction methods 
assume an exponential lapse rate for the modulus of 
refraction. This does cause some increase in the 'errors' in 
the results, mostly at the lower elevation angles. Again, it 
must be an operational decision as to how much error is 
'allowable' for any specific mission. By noting the amount 
of 'error' shown in the tabulated results for atmospheric 
conditions most closely matching those of the test day, 
decisions can be made as to the lowest useable elevation 
angles for tracks at specific altitudes. 

For orbital tracks, if extremely large errors are to be 
avoided, one of the 'exact' methods must be used. After each 
Shuttle mission, data from all participating radars in the 
world-wide NASA/DoD tracking network are analyzed. Bias 
errors of 30 to 40 meters are considered excessive. Since 
many of the orbital tracks are performed at fairly low 
elevation angles, neither the Cyber nor the Varian methods 
would provide corrections suitable for post-mission analysis. 
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APPENDIX A 

TABULATED REFRACTION COMPARISON DATA 
FOR SEA-LEVEL MODULUS OF REFRACTION OF 0.0002550 
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COMPARISON OF GRADIENT (GMO). WHITE SANDS (CYBER) AND SPHERICAL SLAB (VAR IAN ) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 


SEA 

LEVEL MODULUS 

QF REFRACTION: .000 

2550 

SCALE HEIGHT: 7891 

.85 

TARGET ALTITUDE: 

1000 

METERS 

INPUT DATA ! 

CMD GRAD 

REF SOL 

! CYBER 1 

SPHERICAL SLAB 

1 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 1 

E MEAS 
DEG 

R MEAS ! 

METER i 

E COR 
DEG 

R COR 
METER 

J E COR 
! DEG t 

E COR 
DEG 

R COR 
METER 

1 NO CORR 
I METER 

1 

i 

GMD H 
METER 


CYB H : 

METER J 

VAR H : 

METER J 

0.0 

■ 

126366.26 ! 

-.1139 

126335.27 

{ -.1131 

-.1158 

126328.15 

• 

\ 

1252 

1 

• 

1000 


1002 

1 

996 ! 

.5 

74598.29 ! 

.4339 

74580.28 

: .4358 

.4315 

7 4579.37 

» 

i 

1087 

■ 

1001 


1004 i 

998 { 

1 . 0 

48756.97 1 

.9572 

48745.25 

J .9585 

.9574 

48745.04 

i 

■ 

1037 

■ 

1001 


1002 i 

1001 J 

1.5 

35232.32 1 

1.4693 

35223.87 

! 1.4702 

1 .4698 

35223.83 

1 

1020 

i 

1000 


1001 

1001 : 

2.0 

27318.02 J 

1.9763 

27311,47 

1 1.9769 

1.9765 

27311.48 

i 

1012 

i 

1000 


1001 

looo ; 

2 . S 

22220.80 J 

2.4809 

22215.48 

i 2.4812 

2.4806 

22215.49 


1008 

\ 

1000 


1001 

1000 s 

3.0 

18693.10 1 

2.9840 

18688.62 

! 2.9842 

2.9835 

18688.64 


1006 

■ 

1000 


1000 

1000 1 

3 , S 

16117.68 1 

3.4863 

16113.82 

I 3,4864 

3.4857 

16113.83 


1004 

i 

• 

1000 

1 

1000 

1000 

4, 0 

14159.13 I 

3.9881 

14155.74 

1 3.9881 

3.9877 

14155.75 


1003 

1 

1000 

! 

1000 

1000 

4.5 

12621.62 l 

4 .4894 

12618.60 

i 4.4894 

4 .4894 

12618.60 


1003 

i 

■ 

1000 

1 

1000 

1000 

S. 0 

11383.65 I 

4.9905 

11380.92 

{ 4.9904 

4.9901 

11380.93 


' 1002 

• 

1000 

• 

• 

1000 

1000 

6.0 

9515.18 } 

5.9922 

9512.90 

i 5.9920 

5.9917 

9512.90 


1002 

i 

1000 

• 

• 

1000 

1000 

7,0 

8173. SO 1 

6.9934 

8171.54 

1 6.9932 

6.9929 

8171.54 


1001 

i 

1000 

t 

1000 

1000 

8.0 

7164.27 J 

7.9943 

7162.55 

J 7.9940 I 

7.9937 

7162.55 

J 

1001 

i 

i 

1000 

1 

1 

1000 

1000 

9.0 

6378.03 1 

8,9950 

6376,50 

\ 8.9947 

8.9944 

' 6376.50 


1001 

: • 

1000 

1 

1000 

1000 

10.0 

S748.54 J 

9.9955 

5747.16 

! 9.9952 ! 

9.9950 

S747.16 


1001 

• 

• 

1000 

1 

1000 

1000 

12.0 

4804,21 1 

11.9964 

4803.06 

1 11.9960 

11,9958 

4803.06 


1001 

• 

1000 

1 

1000 

1000 

14.0 

4130,38 I 

13.9970 

4129.39 

I 13.9966 ! 

1 13.9964 

4129.39 

1 

1000 

i 

1000 

1 

1000 i 

: iooo 

16.0 

3626.05 J 

15.9975 

3625.18 

1 15.9971 

! 15.9969 

3625.18 

\ 

1000 

• 

• 

1000 

i 

i 

1000 

t 1000 

18.0 

3234.92 i 

17.9978 

3234.15 

J 17.9974 1 

! 17.9973 

3234,15 

i 

* 

1000 

; 

1000 

• 

i 

1000 

; iooo 

20.0 

2923.12 ! 

19.9981 

2922.42 

1 19.9977 

S 19.9976 

2922,42 

l 

1000 

< 

1000 

• • 

1000 

! 1000 

25.0 

2366.08 i 

24.9986 

2365.51 

S 24.9982 

1 24.9981 

2365.51 

* 

i 

1000 

i 

i 

1000 

• 

1000 

\ 1000 

■30.0 

• 2000.10 : 

29.9990 

1999.62 

1 29.9985 

! 29.9985 

1999.62 

* 

• 

1000 

• • 
i 

1000 

; 

1000 

i 1000 

35.0 

1743.64 ! 

34.9992 

1743.22 

{ 34.9988 

I 34.9987 

1743.22 

i 

i 

1000 

i 

1000 

■ 

1000 

i 1000 

40.0 

1555,96 ! 

39.9994 

1555.59 

J 39.9990 

l 39.9989 

1555.59 

i 

« 

1000 

i 

1000 

i 

1000 

: iooo 

45,0 

1414.46 ! 

44,9995 

1414.12 

1 44.9992 

! 44.9991 

1414.12 

• 

1000 

i 

1000 

» 

1000 

I 1000 

50,0 

1305.66 i 

49.9996 

1305.35 

» 49.9993 

! 49.9993 

1305.35 

• 

1000 

i 

1000 

• 

1000 

; iooo 

60.0 

1154.95 1 

59.9998 

1154.67 

J 59.9995 

5 59.9995 

1154.67 

1 

1000 

i 

i 

1000 

1 

1000 

; iooo 

70.0 

1064.42 ! 

69.9999 

1064.17 

1 69.9997 

I 69.9997 

1064.17 

« 

1000 

i 

1000 

1 

1000 

i 1000 

80.0 

1015.67 1 

79.9999 

1015.43 

I 79.9999 

i 79.9998 

1015.43 

J 

1000 

• 

• 

1000 

i 

i 

1000 

: iooo 

90.0 

1000.24 1 
l 

90,0000 

1000,00 

: 90.0000 

1 90.0000 
• 

1000.00 

\ 

\ 

1000 

i 

i 

% 

1000 

l 

■ 

1000 

J 1000 

s 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 
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COMPARISON OF GRADIENT (GMD) . UNITE SANDS (CYBER) AND SPHERICAL SLAB ( VARIAN) CORRECTION METHODS 


BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0002550 SCALE HEIGHT! 7QV1.QS TARGET ALTITUDE! 2000 METERS 


J • INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB J 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

: E MEAS 

R MEAS 

E COR 

R COR 

E COR 

E COR 

R COR i 

NO CORR 


GMD H 


CYB H S 

VAR H 

: DEG 

METER 

DEG 

METER 

DEG 

DEG 

METER 

METER 


METER 


METER 

METER 

1 0.0 

178270.95 

-.1578 

178228.84 

-.1596 

-.1602 

1 

178223.81 I 

2491 


1999 


1994 S 

1991 

,s 

121801.79 ! 

.3948 

121773.74 

.3976 

.3900 

121772.27 ! 

2226 


2001 


2008 S 

1991 

1.0 

87098.70 

.9256 

87078.86 

.9287 

.9265 

87078.41 S 

2114 

• 

2001 


2006 : 

2002 

i . s 

65906.59 

1.4441 

65891.65 

1.4467 

1 .4458 

65891 . 57 ! 

2065 


2001 


2004 S 

2003 

■ 2.0 

52345.88 

1.9550 

52334.04 

1.9579 

1.9569 

52334.07 \ 

2041 


2001 


2003 S 

2001 

2.5 

43157.26 

2.4637 

43147.51 

2.4654 

2.4640 

43147.57 1 

2028 


2000 


2002 S 

2001 

3.0 

36602.46 

2.9694 

36594.20 

2.9707 

2.9691 

36594.26 S 

2020 


2000 


2002 1 

2000 

3,5 

31725.27 

3.4735 

31718.11 

3.4747 

3.4731 

31718.16 1 

2015 


2000 


2001 S 

2000 

4.0 

27969.00 

3.9768 

27962.69 

! 3.9777 

3.9766 

27962.73 1 

2012 

. 

2000 


2001 S 

2000 

4.5 

24994.22 . 

4.4793 

24988.59 

4.4801 

4.4799 

24988.61 i 

2010 


2000 


2001 I 

2000 

5.0 

22583.80 

4.9814 

22578.71 

4.9820 

4.9814 

22578.73 S' 

2008 


2000 


2001 S 

2000 

6.0 

18922.68 

5.9845 

18918.42 

5.9850 

5.9843 

18918.42 ! 

2006 


2000 


2001 S 

2000 

7,0 

16278.67 

6.9868 

16275.00 

6.9871 

6.9865 

16275.00 1 

2004 


2000 


2001 ! 

2000 

8.0 

14282.54 S 

7.9885 

14279.32 

! 7.9887 

7.9881 

14279.32 i 

2003 1 


2000 


2001 S 

2000 

9.0 

12723.67 I 

1 8.9898 

12720.80 

! 8.9900 

8.9894 

12720.80 ! 

2003 


2000 


2001 i 

2000 

10.0 

11473.44 I 

9.9909 

11470.86 

9.9910 

9.9904 

11470.85 ! 

2002 


2000 


2001 S 

2000 

12.0 

9594.73 1 

11.9925 

9592.57 

! 11.9925 

! 11.9920 

9592 . 57 ! 

2002 

1 

2000 


2001 S 

2000 

14.0 

8252.16 I 

13.9937 

8250.30 

13.9936 

13.9932 

8250.30 I 

2001 


2000 


2001 ; 

2000 

16.0 

7246.36 ! 

15.9946 

7244.73 

1 IS. 9944 

! 15.9941 

7244.73 J 

2001 


2000 


2000 S 

2000 

18.0 

6465.82 J 

17.9953 

6464.36 

17.99S1 

17.9948 

6464.36 i 

2001 


2000 


2000 l 

2000 

20.0 

5843.33 I 

19.9958 

5842.01 

1 19.9956 

I 19.9953 

5842.01 S 

2001 

• 

2000 


2000 ! 

2000 

25.0 

4730.71 ! 

24.9969 

4729.64 

24.9966 

24.9963 

4729.64 S 

2001 


2000 


2000 1 

2000 

30 . 0 

3999.38 1 

1 29.9975 

•3998.48 

29.9972 

l 29.9970 

3998.48 J 

2001 

i • 
i 

2000 


2000 { 

2000 

35.0 

3486.78 ! 

34.9980 

3486.00 

34.9977 

34.9976 

3485.99 J 

2001 


2000 


2000 I 

2000 

40.0 

3111.59 \ 

39.9984 

3110.89 

39.9981 

! 39.9980 

3110.89 S 

2001 

• 

i 

2000 


20 0 0 S 

2000 

45.0 

2828.70 { 

44.9987 

2828.06 

44.9984 

44.9983 

2828.06 i 

2001 


2000 


2000 i 

2000 

50.0 

2611.17 1 

49.9989 

2610.58 

49.9987 

! 49.9986 

2610.58 S 

2000 

i 

i 

2000 


2000 ; 

2000 

60.0 

2309.82 { 

59.9993 

2309.30 

S9.9991 1 

59.9990 

2309.30 S 

2000 1 

1 

2000 


2000 ! 

2000 

70 . 0 

2128.80 1 

69.9996 

2128.32 

1 69.9994 I 

1 69.9994 

2128.32 ! 

‘ 2000 

» 

i 

2000 


2000 ; 

2000 

80.0 

2031.30 J 

79.9998 

2030.84 

79.9997 ! 

79.9997 

2030.84 i 

2000 ! 


2000 


2000 : 

2000 

90 . 0 

2000.45 i 

90.0000 

2000.00 

; 90.0000 : 

1 90.0000 

2000.00 S 

2000 ! 

• 

t 

2000 


2000 ; 

2000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 
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COMPARISON OF GRADIENT (GMD) , WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 


BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE’ 


SEA 

LEVEL MODULUS 

OF REFRACTION! .000 

2550 

SCALE HEIGHT 1 7891.85 

TARGET ALTITUDE! 

5000 

METERS 

l 

{ INPUT DATA \ 

GMD GRAD 

REF SOL 

! CYBER ! 

SPHERICAL SLAB \ 

ALTITUDE 

COMPUTED BY EACH SOLUTION METHOD ! 

{ E MEAS 
1 DEG 

R MEAS ! 

METER ! 

E COR 
DEG 

R COR 
METER 

! E COR i 
i DEG ; 

E COR 
DEG 

R COR ! 

METER I 

NO CORR : 
METER 1 

GMD H 1 

METER ! 

CYB H 
METER 

VAR H 
METER 

1 *■ — “ - 

: o.o 

1 

280030.55 ! 

- .2346 

279970.97 

: ( 
1 -.2507 i 

- , 1969 

i 

279971.38 \ 

1 

6144 i 

4996 1 

4920 

5181 

i .5 

219219,81 1 

.3236 

219175.28 

i .3243 i 

.3237 

219173.76 1 

5678 1 

5002 : 

5006 

5002 

1 ' 1.0’ 

174242.73 1 

.8632 

174208.28 

1 .8685 

.8663 

174207.65 i 

5419 t 

5001 i 

5019 

5011 

i 1.5 

141525.93 1 

1.3905 

141498.38 

i 1.3968 1 

1.3945 

141498.29 » 

5273 ! 

5001 \ 

5018 

5011 

; 2 .o • 

117614.62 J 

1.9098 

117591.94 

S 1.9159 

1.9128 

117592.08 i 

5187 

5001 1 

5015 

5007 

i 2.5 

99838.68 ! 

2.4239 

99819.54 

1 2.4295 1 

2,4254 

99819.74 \ 

5134 

5001 { 

5012 

5003 

! 3.0 

86324.31 \ 

2.9346 

86307.82 

! 2.9396 

2.9349 

86303.02 ! 

5100 

5001 S 

5009 

5001 

! 3.5 

7 5816.91 « 

3.4427 

75802.47 

! 3.4472 i 

3.4427 

75802.63 ! 

5077 

S000 .1 

5007 

5000 

! 4*. 0 

67467,16 i 

3.9492 

67454.33 

1 3,9532 

■ 3,9497 

67454.45 \ 

5061 

5000 1 

■5006 

5001 

: 4.5 

60702.41 1 

4.4544 

60690.88 

! 4. 4581 

4.4565 

60690.97 i 

5050 

5000 1 

5005 

5003 

1 '5.0 

55127.38 ! 

4.9587 

55116,92 

J * 4.9620 

4.9608 

55116.97 \ 

5041 

5000 S 

5005 

5002 

1 6,0 

46510.47 ! 

5.9653 

46501.66 

: 5.9681 

5.9664 

46501.66 1 

SO 29 

5000 ! 

5004 

5001 

{' 7 . 0 

40185,42' 1 

6.9702 

40177.81 

S 6.9725 I 

6.9707 

40177.80 1 

5022 

5000 

5003 

5001 

; 8.o 

35359.66 ! 

7.9739 

35352.97 

i 7,9759 i 

7.9741 

35352.95 1 

5017 

5000 \ 

5002 

5000 

» 9.0 

31563.76 ! 

8.9768 

31557.79 

J 8.9786 I 

8.9768 

31557.77 i 

5014 

5000 i 

5002 

5000 

1 10. 0 

28503,80 J 

9.9792 

28498.41 

! 9.9807 ! 

9.9791 

28498.39 J 

5011 

5000 J 

5002 

5000 

! 12.0 

23882.29 S 

11.9Q2B 

23877.78 

1 11.9840 ! 

11.9825 

23877.77 ! 

5008 

5000 : 

5002 

5000 

! 14.0 

20564.55 1 

13,9853 

20S60 .66 

\ 13.9863 ! 

I 13.9850 

20560.66 J 

5006 

; 5000 i 

5001 

5000 

' 16.0 

18071.90 < 

15.9873 ’ 

18068.49 

i 15.9881 

! 15.9869 

18068,48 S 

5005 

! 5000 i 

5001 

5000 

J 18.0 

16133,78 I 

17.9888 

16130.73 

! 17.9895 i 

! 17.9894 

16130.73 S 

5004 

! 5000 1 

5001 

5000 

! 20.0 

14586,01 I 

19.9901 

14583.26 

\ 19.9906 

1 19.9897 

14583.25 ! 

5003 

1 5000 ! 

5001 

5000 

J 25.0 

11815.57 ! 

24.9923 

11813.34 

i 24.9927 i 

1 24.9919 

11813.34 1 

5002 

1 5000 i 

5001 

5000 

J 30.0 

9992.13 I 

29.9938 

9990.24 

J 29.9941 

■ 29.9935 

9990.24 ! 

5002 

: 5000 ! 

5001 

5000 

1 35.0 

* 8713.10 S 

34.9950 

8711.45 

! 34.9951 ! 

! 34.9946 

8711.45 i 

5002 

S 5000 i 


5000 

I 40.0 

7776.49 1 

39.9958 

7775.02 

1 39.9959 

! 39.9955 

7775.02 ! 

5001 

I 5000 1 

5001 

5000 

! 45.0 

7070.10 l 

44.9965 

7068.77 

! 44.9966 I 

! 44.9962 

7068.76 i 

5001 

i 5000 ; 

5001 

5000 

: 50.0 

6526.77 1 

49.9971 

6525.54 

{ 49.9971 

! 49.9968 

6525.54 1 

5001 


5001 

5000 

I 60.0 

5773.97 \ 

59.9980 

5772.88 

\ 59,9980 I 

i 59.9978 

5772.88 J 

5001 

vf‘.& 


5000 

: 70.0 

5321.66 J 

69,9988 

5320.66 

1 69.9988 

! 69.9986 

5320.66 ! 

5001 


5001 

5000 

! 80.0 

5078.04 J 

79,9994 

5077, 08 

! 79.9994 

i 79,9993 

5077.08 ! 

5001 


S001 

5000 

! 90.0 

5000,94 S 

90.0000 

5000.00 

i* 90 . OQOO 
1 

! 90.0000 
» 

1 

5000.00 1 

1 

5001 


5001 

5000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 
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COMPARISON OF GRADIENT ( GMD ) , UNITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHOD 


BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' - 
SEA LEVEL MODULUS OF REFRACTION t .0002550 SCALE HEIGHT! 7891.8S TARGET ALTITUDE : 10000 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

i ■ SPHERICAL SLAB !' 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 
DEG 

R MEAS 
METER 

E COR 
DEG 

R COR 
METER 

i E COR 
DEG 

! ‘ E COR 
DEG 

R COR : 

METER ! 

NO CORR 
METER 



GMD H 
METER 



CYB H 
METER 

■ 

■ 

< 

* 

VAR H 
METER 

0.0 

392550.88 

-.3030 

392479.09 

-.3514 

! -.2132 

i 

392484.97 \ 

1206? 



9992 



9665 

■ 

« 

i 

i 

10607 

.5 

329604.02 

• 2602 

329547.82 

.2492 

.3012 

329547.82 I 

11383 



10002 



9942 

i 

1 

10238 

1.0 

278818.24 

.8052 

278772.8? 

.8078 

\ .8307 

278772.82 ! 

10951 



10002 



10017 

1 

1 

10126 

1.5 

238218.31 

1.3382 

238180.78 

i • 34S8 

1.3552 

238180.94 1 

10676 



10002 



10035 

i 

* 

10072 

2.0 

205315.75 

1.8629 

205784.03 

1.8721 

1.8740 

205784.34 i 

10495 



10001 



10037 

i 

i 

10041 

2.5 

179863.41 

2.3818 

179836.11 

2.3914 

2.3885 

179836.45 S 

10373 



10001 



10033 

i 

i 

10022 

• 3.0 

158912.11 

2.8966 

158888. 2S 

2.9059 

2.9003 

158888.55 1 

10288 



10001 



1 0028 

i 

i 

10011 

3.5 

141840.65 

3.4084 

141819.52 

3.4172 

3.4107 

141819.75 J 

• 10228 



10001 



10024 

t 

i 

10006 

4.0 

127768.13 

3.9180 

127749.20 

3.9262 

3.9205 

■ 127749.36 ! 

101B4 



10001 



1 0020 

i 

i 

1 00 06 

4.5 

116035.59 

4.4259 

116018.47 

4.4335 

4.4304 

116018.57 \ 

10151 



10000 



10017 

• 

• 

10009 

5.0 

106146.01 

4.9325 

106130.41 

4.9396 

• 4.937S 

106130.44 i 

10126 



10000 



10015 

1 

1 

10010 

6.0 

90480.66 

5.9428 

90467.42 

5.9490 

5,9458 

90467.37 ,* 

10092 



10000 



10012 

; 

100 05 

7.0 

78703.74 

6.9506 

78692.26 

6.9560 

6.9524 

78692.18 i 

1006? 



10000 



10009 

i 

i 

1 0003 

8.0 

69571.96 

7.9565 

69561.83 

7.9613 

7.9576 

69561.75 ! 

10054 



10000 



1 0008 

i 

• 

100 02 

9.0 

62306.92 

1 8.9612 

62297.86 

l 8.9655 

8.9619 

62297.79 i 

10043 



10000 



10006 

* 

i 

100 01 

10. 0 

56401.90 

9.9651 

56393.71 

9.9689 

9.9655 

56393.65 S 

10036 



10000 

1 


10005 

i 

* 

1000 1 

12.0 

• 47409.00 

11.9709 

47402.13 

! 11.9741 

11.9710 

47402.09 *, 

10025 



10000 



10004 

i 

i 

1 0 0 0 0 

14.0 

40903.87 

13.9752 

40897.94 

; 13.9779 

13.9751 

40897.92 I 

10019 



10000 



10004 

i 

i 

100 00 

16.0 

35992.80 

15.9784 

35987.59 

! 15.9807 

15.9783 

35987.57 ! 

10015 



10000 



1 00 03 

i 

i 

1 00 0 0 

18.0 

32161.78 

17.9810 

32157.12 

17.9830 

17.9808 

32157.12 1 

10012 



10000 



1 0003 

i 

i 

1000 0 

20 , 0 

29095.31* 

19.9830 

29091 .10 

! 19.9848 

19.9828 

29091.09 1 

10010 

i 


10000 



10002 

i 

i 

1 00 0 0 

25.0 

23592.69 

24.9868 

23589.28 

24.9881 

24.9865 

23589.27 l 

10006 

i 


10000 



10002 

i 

i 

1000 0 

30 . 0 

19962.69 

29.9894 

19959.80 

1 29.9904 

29.9891 

19959.80 I 

10005 



10000 



10002 

i 

i 

10000 

35.0 

17413.13 

34.9913 

17410.61 

34.9921 

34.9910 

17410.61 S 

10004 

1 


10000 



1 0002 

i 

i 

1 0000 

40.0 

15544,65 

39.9927 

15542.40 

! 39.9934 

39.9925 

15542.40 1 

10003 



10000 



10002 

i 

i 

1000 0 

45.0 

14134.68 

44.9939 

14132.64 

44.9945 

44.9937 

14132.64 I 

10003 

1 


10000 



1 0002 

i 

< 

100 00 

50.0 

13049.78 

49.9949 

13047.89 

! 49.9953 

49.9947 

13047.8? ! 

10002 



10000 



10002 

i 

i 

10000 

60.0 

11546.09 

59 . 9965 

11544.42 

59.9968 

59.9963 

11544.42 I 

10002 

i 


10000 



1 000 1 

i 

« 

100 00 

70.0 

10642.37 

69.9978 

10640.83 

1 69.9980 

69.9977 

10640.83 1 

10002 



* 10000 



10001 

t 

i 

1 00 0 0 

80.0 

10155.52 

79.9989 

10154.05 

79.9990 

79.9989 

10154.05 ; 

10001 



10000 



10001 

r 

i 

10000 

90.0 

10001.45 

90 .0000 

10000.01 

[ 90,0000 

90.0000 

10000.00 { 
« 
1 

10001 


I 

10000 



10001 

i 

i 

i 

i 

1000 0 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD). WHITE SANDS (CYBER > AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION I .0002550 SCALE HEIGHT! 7891.85 TARGET ALTITUDE: 20000 METERS 


INPUT 

DATA 1 

GMD GRAD 

REF SOL 

! CYBER ! 

SPHERICAL SLAB S 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 
DEG 

R MEAS J 

METER ! 

E COR 
DEG 

R COR 
METER 

1 E COR 1 
S DEG 1 

E COR 
DEG 

R COR : 

METER J 

NO CORR 
METER 


GMD H ! 

METER ! 


CYB H ! 

METER i 

VAR H 
METER 

0.0 

« 

548278.59 ! 

-.3702 

548199.26 

• i 

i , 

! -.4908 ! 

-.2498 

• 

i 

548209.40 \ 

23522 


1 

19985 J 


18841 ! 

21134 

.5 

483896,25 J 

.1992 

483832,94 

1 .1559 5 

.2694 

483834.07 ! 

22539 


20002 ; 


19642 i 

20593 

1.0 

428976.01 1 

.7496 

428923.95 

{ .7368 ! 

.7963 

428924.33 1 

21875 


20002 1 


19911 ! 

20351 

i.S 

382189.74 ! 

1.2873 

382145.91 

J 1.2880 { 

1.3193 

382146.18 ! 

21419 


20002 ; 


20010 i 

20214 

2.0 

342318.67 S 

1.8163 

342281.06 

! 1.8232 1 

1.8380 

342281.33 \ 

21098 


20002 


20046 i 

20130 

2.5 

308314.95 J 

2.3392 

308282.16 

1 2.3488 J 

2.3533 

308282.41 l 

20866 


20001 : 


20056 ! 

20077 

3.0 

279249.06 

2.8575 

279220.10 

1 2.8683 l 

2,8663 

279220.28 I 

20694 


20000 1 


20055 i 

20043 

3.5 

254352.52 ! 

3.3724 

254326.65 

! 3.3835 ! 

3.3784 

254326.74 l 

20566 




20052 1 


4.0 

232927.71 

3.8848 

232904.38 

! 3.8958 1 

3.8903 

232904.39 1 

20469 

» 

• 

20000 


20047 : 

20023 

4.5 

214401,83 ! 

4.3951 

214380.61 

! 4.4058 ! 

4.4027 

214380.55 J 

20393 


20000 


20042 : 

20028 

5.0 

198298.26 

4.9039 

198278.83 

! 4,9141 { 

4.9113 

198278 * 69 ! 

20333 

5 

20000 


20038 i 

20026 

6.0 

171854.02 . 

5.9179 

171837,42 

! 5.9272 J 

5.9224 

171837.21 J 

20247 


20000 


20030 *, 

20014 

7.0 

151215.66 

6,9285 

151201,19 

1 6.9369 ! 

6,9314 

151200.97 i 

20189 

1 

1 

20000 


20024 \ 

20008 

8.0 

134774.43 1 

7,9368 

134761.62 

1 7.9444 1 

7.9388 

134761.42 1 

20149 

i 

20000 


20020 1 

20005 

9.0 

121431.30 1 

8.9435 

121419.82 

1 8,9504 1 

8.9448 

121419.65 1 

20120 

\ 

20000 


20017 1 

20003 

10. 0 

110422.65 l 

9.9489 

110412,25 

! 9.9552 1 

9,9499 

110412.12 : 

20099 

\ 

20000 


20014 : 

20002 

12.0 

93394.58 ! 

11,9573 

93385.82 

J 11.9627 J 

11.9578 

93385.75 1 

20070 

\ 

20000 


20011 ! 

20001 

14,0 

80896.17 S 

13.9635 

80888.61 

l 13.9680 l 

13.9637 

80888.56 \ 

20052 

1 

20000 


20008 I 

20000 

16.0 

71370.24 I 

15.9682 

71363.58 

! 15.9722 1 

I 15.9682 

71363,55 I 

20040 

I 

20000 


20007 { 

20000 

18.0 

63890.66 J 

17,9719 

63884.71 

l 17,9754 ! 

17.9718 

63884.69 ! 

20032 

t 

20000 


20006 ; 

20000 

20.0 

57875.86 i 

1 19.9748 

57870.48 

J ' 19.9780 1 

t 19.9748 

57870.46 i 

20026 

, 

20000 


20005 ! 

20000 

25.0 

47027.34 1 

24.9804 

47022.97 

i 24.9828 { 

24,9802 

47022.96 ! 

20017 

, 

20000 


20004 ; 

20000 

30.0 

39836.95 ! 

1 29.9841 

39833.25 

l 29.9861 ! 

29,9840 

39833.25 I 

20011 

i 

20000 


20003 l 

20000 

35.0 

34773.07 I 

! 34.9869 

34769.85 

! 34.9885 j 

34.9868 

34769.84 « 

20008 

, 

, 

20000 


20003 : 

20000 

40.0 

31055.67 ! 

! 39.9891 

31052.79 

{ 39.9904 ! 

39,9889 

31052.79 ! 

20006 

■ 

20000 


20002 ; 

20000 

45.0 

28247.36 J 

! 44,9909 

28244.74 

! 44.9920 J 

44,9907 

28244.74 [ 

20005 

, 

20000 


20002 ! 

20000 

50 . 0 

26084.85 ! 

I 49.9924 

26082.43 

r 49,9933 : 

49.9922 

26082.43 J 

20004 

• 

i 

20000 


20002 s 

20000 

60.0 

23085.37 ! 

i S9.994B 

23083.23 

S S9.99S4 i 

59.9946 

23083.23 1 

20003 

■ 

• 

20000 


20002 J 

20000 

70.0 

21281.58 

1 69.9967 

21279.61 

S 69,9971 

69.9966 

21279.61 \ 

20002 

, 

20000 


20002 5 

20000 

80.0 

20309.53 ! 

! 79,9984 

20307.65 

i 79.9986 ! 

79.9984 

20307.65 ! 

20002 

i 

20000 


20002 I 

20000 

90.0 

20001.85 

[ 90.0000 
1 
« 

20000.00 

1 90.0000 
i ! 

90.0000 

20000,00 ! 
« 

20 Q 02 

* 

• 

, 

» 

20000 

i 

20002 ! 
, 
i 

20000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 














A-7 


COMP ARISON OF GRADIENT <GHD) % WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 


BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0002550 SCALE HEIGHT: 7891.85 TARGET ALTITUDE: 50000 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB 


ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

HETHOD ! 

E MEAS 
DEG 

R MEAS 
METER 

E COR 
DEG 

R COR 
METER 

E COR 
DEG 

E COR 
DEG 

R COR 
METER 


NO CORR 
METER 



GMD H 
METER 



CYB H 
METER 

l 

i 

i 

VAR H 1 

METER \ 

0.0 

850619.23 

-.4406 

850536.97 

* 

-.7615 

-.3200 

B50S49 . 26 


56471 



49973 



45256 

i 

51752 J 

.5 

7 8S027.35 

.1383 

784961.58 

.0050 

.1981 

784962.12 


54925 



50002 



48197 

! 

50814 ! 

1.0 

726547.93 

.6959 

726493.68 

.6340 

.7353 

726493.54 


53834 



50002 



49231 

i 

50499 ! 

•1.5 

674119.01 

1.2393 

674073.18 

! 1.2101 

1.2661 

674072.99 


53050 



50002 



49668 

i 

50315 1 

2.0 

626895.67 

1.7729 

626856.21 

1.7604 

1.7906 

626856, 08 


52471 



50002 



49871 

i 

50194 

2.5 

584253.33 

2.2996 

584218.81 

2.2960 

• 2.3105 

584218.71 


52032 



50002 



49970 

1 

50112 

3.0 

545672.56 

2.8211 

545641.98 

2.8226 

2.8272 

545641.84 


51691 



49999 



50017 

t 

50056 

3.5 

510767.64 

3.3389 

510740.24 

3.3432 

3.3426 

510740.05 


S1426 



49999 



50041 

i 

50032 

4.0 

479139.08 

3.8537 

479114.31 

3.8596 

3.8577 

479114.06 


51214 



49999 



50052 

S 

50033 

4.5 

450449,96 

4.3661 

450427.38 

4.3730 

4.3734 

450427.08 


51044 



50000 



50056 

. 

50056 

5.0 

424395.85 

4.8768 

424375.13 

4.8841 

4.8837 

424374.78 


50905 



50000 



50056 

! 

50050 

6.0 

379119.42 

5.8940 

379101.66 

5.9015 

5.8982 

379101.27 


50695 



50000 



50052 

! 

50027 

7.0 

341423.78 

6,9073 

341408.27 

6.9145 

6.9099 

341407.90 


50547 



50000 



50045 

{ 

SO 0 15 

8.0 

309798.46 

7.9177 

309784.70 

7.9246 ! 

7.9195 

309784.38 


50440 



50000 



50039 

J 

50009 

9.0 

283047.94 

8.9261 

283035.59 

8.9326 { 

8.9274 

283035.33 


50360 



50000 



50034 

i 

i 

50006 

10,0 

260232.76 

9,9331 

260221.56 

9.9391 1 

9.9340 

260221.35 


50299 



50000 



50029 

• 

• 

50004 

12.0 

223617.64 

ii.9439 

223608.20 

11.9491 i 

11.9443 

223608.07 


50215 



50000 




\ 

50002 

14.0 

195742.91 

13.9518 

195734.75 

13,9564 ! 

13.9521 

195734.66 


50161 



50000 



50018 

i 

5000 1 

16.0 

173956.67 

15.9579 

173949.40 

15.9620 ! 

15.9580 

173949.42 


50124 



50000 



50014 

* 

• 

50001 

18.0 

1S6541 .85 

17.9627 

156535.42 

17.9664 ! 

17.9628 

156535.38 


50098 



50000 




■ 

• 

50000 

20.0 

142352.81 

19.9667 

142346.99 

19.9699 S 

19.9667 

142346.96 


50079 



M'M'll 



50010 

• 

i 

50000 

25.0 

116380 .39 

24.9739 

116375.67 

24.9765 ! 

24.9739 

116375.65 


50050 



50000 



50007 

i 

50000 

30.0 

98925.13 

29.9789 

98921 . 13 

29,9810 1 

29.9788 

98921.12 


50033 



50000 



50005 

■ 

i 

50000 

35.0 

86531. 9S 

34.9026 

86528.46 

34.9843 i 

34.9825 

86528.45 


50024 



50000 



50004 

i 

50000 

40 . 0 

77387.40 

39.9855 

77384.28 

39.9869 i 

39.9854 

77384.28 


50017 



50000 



50004 

i 

soooo 

45.0 

70455,48 

44.9878 

70452.65 

44.9890 ! 

44.9877 

70452.64 


50013 



50000 



50003 

i 

i 

50000 

50.0 

65104.91 

49.9898 

65102.29 

49.9908 1 

49.9897 

65102.29 


50009 



50000 



50003 

• 

■ 

50000 

60 , 0 

57666.77 

59.9930 

57664.45 

59.9936 ! 

59.9929 

57664.45 


50006 



50000 



50002 

i 

i 

50000 

70.0 

53185.14 

69.9956 

53183.01 

69.9960 J 

69.9955 

53183.00 


50003 



50000 


1 

50002 

• 

i 

50000 

80.0 

S0767.57 

79.9979 

50765.53 

79.9981 ! 

79.9978 

50765.53 


50002 



50000 



50002 

• 

i 

50000 

90.0 

50002.01 

90.0000 

50000.00 

90.0000 ! 

90.0000 

50000.00 


50002 



50000 



50002 

• 

SOOOO 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 













COMPARISON OF GRADIENT (GMD) . WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION: , 0002S50 SCALE HEIGHT: 7B91.8S TARGET ALTITUDE: 100000 METERS 


\ INPUT 

DATA ! 

GMD GRAD 

REF SOL i 

CYDER i 

SPHERICAL SLAB i 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

1 E MEAS 

R MEAS i 

E COR 

R COR i 

E COR ! 

E COR 

R COR ! 

NO CORR 

GMD H J 

CYB H 1 

VAR H 

1 DEG 

METER ! 

DEG 

METER 1 

DEG i 

DEG 

METER i 

METER 

METER l 

METER J 

METER 

0.0 

; 

1188162.14 1 

- . 4778 

J 

1188079.25 ! 

l 

-1.0637 i 

-.3634 

\ 

1188092.14 ! 

109725 

i 

99963 ' 

88005 : 

102300 

.5 

1121980.82 ! 

.1080 

1121914.68 i 

-.1278 i 

. 1520 

1121914.52 1 

107568 

100003 S 

9S467 ! 

100851 

i.O 

1061752.84 ! 

.6705 

1061698.37 J 

. 5S50 i 

.6983 

1061697.76 \ 

106022 

100004 1 

97906 { 

iOOSii 

i.5 

1006472.12 

1.2175 

1006426.15 i 

1 . 1555 ! 

1.2358 

100642S . 65 ! 

104894 

100004 

98939 ! 

100319 

2.0 

955381.62 i 

1 .7539 

955342.06 i 

i .7191 

1 .7652 

955341.72 l 

104047 

100003 

99439 1 

100188 

2 , S 

907968.98 

2.2828 

907934.39 

2.2630 

2,2886 

907934.14 ! 

103394 

100003 

99701 

100094 

3.0 

863840.16 ! 

2.8061 

863809.53 1 

2.7952 

2.8082 

863809.28 ! 

102878 

99999 

99842 

100029 

1 3.5 

822750.03 

3.3253 

822722.60 

3.3197 

3.3258 

822722.32 5 

102470 

100000 

99926 

100006 

! 4.0 

784431.18 ! 

3.8413 

784406.38 

3.B391 

3.8429 

784406.06 \ 

102139 


99975 

100022 

: 4.5 

748675.09 

4.3548 

748652.49 

4.3547 

4.3605 

748652.13 i 

101867 

! 100000 

100004 

100074 

! S.O 

715295.01 

4. 86<b3 

715274.27 

4.8677 

4.8720 

715273.87 i 

101641 


100021 

100071 

S 6.0 

6S4992 . 51 

5.8849 

654974.74 

S . 8877 

5.8882 

654974.31 ! 

101293 

1 100000 

100036 

100038 

! 7.0 

602299.32 

6.8992 

602283.80 

6.9027 

6.9012 

602283.39 1 

101040 

! 100001 

100039 

100021 

; 8.o 

556150.08 ! 

7.9105 

556136.32 

7.9141 

7.9118 

556135.97 ! 

100851 

i looooi 

100038 

100013 

J 9.0 

515617.73 ! 

8.9196 

515605.38 

8.9233 

8,9205 

515605.09 i 

100707 

! 100001 

100035 

100008 

1 10. 0 

479901.36 I 

9.9272 

479890 , 16 

9.9307 

9.9277 

479889.92 \ 

100595 

! 100001 

100032 

100005 

: i2.o 

420272.52 1 

11.9388 

420263.08 

11,9421 

11.9391 

420262.93 ! 

100435 

i iooooo 

100026 ! 

100002 

! 14.0 

372917.52 ! 

13.9474 

372909.36 

13.9S04 

13.9476 

372909,26 i 

100330 

i 100000 

100021 

100001 

! 16.0 

334721.29 ! 

15.9541 

334714,10 

15.9567 

15.9541 

334714.03 ! 

100257 

; iooooo 

100017 I 

1 0 0 0 0 1 

1 18.0 

303454,98 ! 

17.9593 

303448.55 

17.9617 

17.9593 

303448.49 i 

100204 

i IOOOOO 

100014 

iooooo 

{ 20.0 

277512.77 ! 

19.9636 

277506.95 

19.9657 

19.9636 

277506.91 1 

100166 

I IOOOOO 

100012 

IOOOOO 

1 25.0 

228998.10 ! 

24.9715 

228993.38 

24.9732 

I 24.9714 

228993.35 » 

100104 

: iooooo 

100009 

iooooo 

! 30.0 

195704.20 ! 

29.9769 

195700.20 

29.9783 

29.9768 

195700.18 i 

1 0 Q 070 

I 100000 

100006 

iooooo 

! 35.0 

171764.65 

34.9809 

171761.16 

34.9821 

34.9809 

171761.15 l 

100048 

; iooooo 

100005 

iooooo 

! 40.0 

153955.67 ! 

39.9841 

153952. 5S 

39.9851 

39.9840 

153952.54 ! 

100034 

: iooooo 

100004 

IOOOOO 

! 45.0 

140381 . 00 

44.9866 

140378.17 

44.9875 

44.9866 

140378.16 l 

100025 

s iooooo 

100004 

IOOOOO 

50.0 

129862.38 ! 

49.9888 

129859.76 

49.9895 

49.9887 

129859.75 1 

100018 

{ iooooo 

100003 

IOOOOO 

: 60.0 

115185.63 

59.9923, 

115183.31 

59.9928 

59.9922 

115183.31 i 

100010 

; iooooo 

100003 

IOOOOO 

I 70.0 

106314.57 ! 

69.9951 

106312.44 

69.9954 

69.9951 

106312.43 ! 

100005 

: iooooo 

100002 

IOOOOO 

! 80.0 

101521 . 08 

79.9976 

101519.04 

79.9978 

79.9976 

101519.04 1 

100003 

; iooooo 

100002 

IOOOOO 

: 90.o 

100002.01 ! 

90.0000 

100000.00 

90.0000 

9 Q .0 000 

100000,00 ! 

t 

1 

100002 

* iooooo 

100002 

IOOOOO 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 














APPENDIX B 

TABULATED REFRACTION COMPARISON DATA 
FOR SEA-LEVEL MODULUS OF REFRACTION OF 0.0003307 


B-l 



COMPARISON OF GRADIENT (GMD) . WHITE SANDS (CYDER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 


BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0003307 SCALE HEIGHT! 6631.54 TARGET ALTITUDE! 1000 METERS 


i INPUT 

DATA 

GMD GRAD 

REF SOL * 

CYBER * 

SPHERICAL SLAB ! 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD { 

i 1 " ■■■ ■* 

J E MEAS 
DEG 

R MEAS 
METER ! 

E COR 
DEG 

R COR l 

METER ! 

E COR 1 
DEG l 

E COR 
DEG 

R COR 
METER ! 

NO CQRR 
METER 

GMD H 
METER \ 

CYB H 
METER 1 

VAR H ! 

METER ! 

! 0.0 

1 

135871.48 

- . 1883 

135828.48 i 

i 

- .2088 J 

-.1913 

i 

135821.22 ! 

1447 

S 

iooo : 

952 ! 

i 

993 ! 

: .s 

77318.02 

.3950 

77294.06 ! 

.3874 ! 

.3912 

77292.22 \ 

1143 

looi : 

991 1 

996 ! 

: i.o 

49618.93 

.9333 

49603.64 1 

.9288 

.9336 

49603. 21 1 

1059 

1001 i 

997 : 

1001 

! 1.5 

35574.52 

1.4525 

35563.58 J 

1.4492 i 

1.4533 

35563.50 » 

1030 

1001 s 

999 . J 

1001 

{ 2.0 

27481.63 

1.9635 

27473,19 

1.9608 

1.9638 

27473.19 \ 

1018 

1000 

999 * J 

1001 

: 2.5 

22310.27 

2.4706 

22303,42 1 

2.4683 ! 

2.4702 

22303.44 i 

1012 

iooo : 

1000 ! 

1000 

! 3.0 

18746.94 

2.9754 

18741.18 

2.9734 

2.9747 

18741.21 J 

1009 

1000 

iooo ; 

1000 

; 3.5 

16152.57 

3.4790 

16147.61 

3.4771 

3.4781 

16147.63 1 

1006 

1000 1 

1000 l 

1000 

\ 4.0 

14183.03 

3.9817 

14178.68 

3.9799 

3.9811 

14178.69 ! 

1005 

1000 

iooo ; 

1000 

: 4.5 

12638.74 

4.4838 

12634.86 ! 

4.4821 ! 

4.4838 

12634.87 ! 

1004 

iooo : 

iooo : 

1000 

! 5.0 

11396,35 

4.9855 

11392.85 

4.9839 

4.9848 

11392.85 l 

1003 

1000 

iooo : 

1000 

6.0 

9522.77 

5.9880 

9519.85 1 

5.9865 1 

5.9873 

9519.85 1 

1002 

1000 

1000 I 

1000 

; 7.0 

8178.45 

6.9899 

8175.94 

6,9885 

6.9891 

8175.94 J 

1002 • 

1000 

iooo : 

1000 

: 8.o 

7167.70 

7.9912 

7165.50 ! 

7.9899 ! 

7.9904 

7165.50 i 

1001 

1000 ! 

iooo : 

1000 

1 9.0 

6380.53 

1 8.9923 

6378,57 1 

8.9910 i 

8.991S 

6378.57 1 

1001 

1000 ' 

1 10 0 0 1 

1000 

! 10.0 

5750.44 

! 9.9932 

5748.68 J 

9.9920 ! 

9.9923 

5748.67 ! 

1001 

1000 J 

iooo : 

1000 

J 12.0 

4B05.40 

} 11.9945 

4803.93 i 

1 11.9933 1 

t 11.9936 

4803.92 J 

1001 

1000 ! 

\ 1000 ! 

1000 

! 14.0 

4131.20 

! 13.9954 

4129.93 ! 

13.9943 J 

13.9946 

4129.93 ! 

1001 

1000 i 

i iooo : 

1000 

! 16.0 

3626.66 

! 15.9961 

3625. SS I 

1 15.9950 

15.9953 

362S.55 ! 

1001 

1000 ! 

: iooo : 

1000 

i 18.0 

3235.39 

I 17.9967 

3234.40 i 

1 17.9956 I 

17.9958 

3234.40 i 

1001 

1000 ! 

; iooo : 

1000 

: 20.0 

2923.49 ; 

1 19.9971 

2922.59 ! 

! 19.9961 

19.9963 

2922.59 J 

1000 

1000 

: iooo : 

1000 

; 25.0 

2366.33 1 

I 24.9979 

2365,60 ! 

24.9970 1 

24.9971 

2365.60 1 

1000 

1000 1 

t iooo i 

1000 

i 30.0 

2000.29 

: 29'. 9984 

1999.68 i 

! 29.9975 

29.9976 

1999.68 ! 

1000 

1000 

iooo ; 

1000 

! 35.0 

1743.79 

! 34.9987 

1743.26 ! 

! 34.9980 1 

34.9981 

1743.25 J 

1000 

1000 I 

iooo : 

1000 

: 40.0 

1556,08 

! 39.9990 

1555.60 ! 

! 39.9983 I 

39.9984 

1555.60 I 

1000 

10Q0 

iooo : 

1000 

1 45.0 

1414.57 

; 44.9993 

1414.14 1 

i 44.9986 ! 

44.9986 

1414.14 1 

1000 

1000 1 

1 iooo s 

1000 

1 so.o 

1305.76 

1 49.9994 

1305 . 36 ! 

1 49.9988 

49.9989 

1305.36 ! 

1000 

1000 

i iooo : 

1000 

! 60.0 

1155 . 03 

! 59.9996 

1154.68 1 

1 59.9992 1 

59.9992 

1154.68 ! 

1000 

1000 

1000 ! 

1000 

: 70.0 

1064.50 

S 69.9998 

1064.18 ! 

! 69.9995 

69.9995 

1064.17 : 

1000 

1000 

1000 ! 

1000 

: ao.o 

101S.74 

1 79.9999 

1015,43 i 

! 79.9997 ! 

79.9998 

1015.43 ! 

1000 

1000 

1000 l 

1000 

i 90.0 

• 

1000.31 

S 90.0000 

1000.01 

1 

! 90.0000 
t ! 

90.0000 

1000.00 ! 

1000 

1000 

1000 i 
1 
1 

1000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT <GHD) . WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 


BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 * REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE* 
SEA LEVEL MODULUS OF REFRACTION: .0003307 SCALE HEIGHT: 6631.54 TARGET ALTITUDE: 2000 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB * 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD ! 

E HEAS 

R MEAS 

E COR 

R COR 

E COR 

E COR 

R COR : 

NO CORR 


GMD H 


CYB H 

VAR H ! 

DEG 

METER 

DEG 

METER 

DEG 

DEG 

METER 

METER 


METER 


METER 

METER : 

0.0 

191171.30 

-.2593 

191113.26 

i 

-.2937 

-.2611 

* 

191110.61 S 

2864 


1998 ' 


i 

1885 

: 

1992 

.S 

127041.86 

.3327 

127004.60 

.3209 

,3242 

127001.83 S 

2373 


2002 


1976 S 

1983 

1. 0 

89279.74 

.8840 

89253.92 

.8782 

.8851 

89253.10 S 

2183 


2001 


1993 

2003 

1.5 

66915.51 

1.4137 

66896.27 J 

1.4100 

1.4162 

66896.12 \ 

2102 


2001 


1997 ! 

2004 

2.0 

52869.41 

1.9322 

52854.25 

1.9294 

1.9338 

52854.29 ! 

2064 


2001 


1999 

2002 

2.5 

43456.83 

2,4445 

43444.39 ! 

2.4422 i 

2.4449 

43444,47 ! 

2043 


2001 


1999 

2001 

3.0 

36787.43 

2.9532 

36776.91 

2.9512 

2.9527 

36776.98 S 

2031 


2000 


2000 

2000 

3.5 

31846.95 

3.4596 

31837.85 

3.4578 

3.4590 

31837.90 I 

2023 


2000 


2000 

2000 

4.0' 

28053.07 

3.9646 

28045.05 

3.9629 

3.9644 

28045.09 8 

2018 


2000 

S 

2000 

2000 

4.5 

25054.65 

4.4685 

25047.49 

4.4669 

4.4694 

25047.51 ! 

2015 


2000 


2000 

2001 

5.0 

22628,67 

4.9716 

22622.21 

4.9701 

4.9717 

22622.21 J 

2012 


2000 

i 

2000 

2000 

6.0 

18949.42 

5.9764 

18944.01 

5.9751 

5.9761 

18944.00 1 

2009 


2000 


2000 

2000 

7.0 

16295.92 

6.9799 

16291.27 

6.9786 

6.9794 

16291.26- ! 

2006 


2000 

1 

2000 

2000 

8.0 

14294.36 i 

7.9825 

14290.28 1 

7.9813 1 

7.9819 

14290.27 i 

2005 


2000 

i 

2000 

2000 

9.0 

12732,14 ! 

8.9845 

12728,51 ! 

! 8.9834 I 

8.9839 

12728.50 S 

2004 


2000 

1 

2000 

20 00 

10,0 

11479.75 ! 

9.9862 

11476.47 J 

9.9850 ! 

9.9855 

11476.47 i 

2003 


2000 

i 

i 

2001 

2000 

12.0 

9598.55 ! 

11.9887 

9595,81 ! 

! 11.9876 ! 

! 11.9879 

9595.81 i 

2003 


2000 

J 

2001 

2000 

14,0 . 

8254,69 ! 

13,9904 

8252.33 ! 

13.9894 ! 

1 13.9897 

8252.33 i 

2002 


2000 


2001 

2000 

16.0 

7248,15 ! 

15.9918 

7246.08 ! 

I 15.9908 1 

[ 15.9910 

7246.08 i 

2002 


2000 

! 

2001 ' 

2000 

18.0 

6467.15 ! 

17.9928 

6465.30 i 

17.9919 ! 

1 17.9921 

6465.30 S 

2001 


2000 

1 

2001 

2000 

20.0 

' 5844.36 1 

19.9937 

5842.69 ! 

1 19.9927 ! 

l 19.9929 

5842.69 i 

2001 


2000 

I 

2001 

2000 

25.0 

4731.33 i 

24.9952 

4729.98 I 

I 24.9943 ! 

I 24.9945 

4729.98 « 

2001 


2000 

• 

< 

2001 

2000 

30 . 0 

3999.80 1 

29.9962 

3998.66 I 

! 29.9954 

I 29.9955 

3998.66 1 

2001 


2000 

i 

2001 

2000 

3S. 0 

3487.10 I 

34.9970 

3486.11 I 

1 34.9962 ! 

! 34.9963 

3486.10 1 

2001 


2000 

• 

2001 

2000 

40.0 

3111.84 ! 

39.9975 

3110.95 ; 

1 39.9968 

! 39.9969 

3110.95 1 

2001 


2000 

• 

2001 

20 0 0 

45.0 

2828.92 I 

44.9980 

2828.11 ! 

1 44.9974 1 

I 44.9974 

2828.11 ! 

2001 


2000 

• 

2001 

2000 

50 . 0 

2611.35 1 

49.9984 

2610.61 

l 49.9978 

! 49.9978 

2610,60 ! 

2001 


2000 

! 

2001 

2000 

60.0 

2309.97 J 

59.9989 

2309.31 I 

1 59.9985 ! 

! 59.9985 

2309.31 1 

2001 


2000 

i 

2001 

2000 

70 . 0 

2128,93 ! 

69.9993 

2128.32 

! 69.9990 

! 69.9991 

2128.32 S 

2001 


2000 

J 

2001 

2000 

80.0 

2031.43 1 

79.9997 

2030.85 ! 

! 79.9995 ! 

! 79.9995 

2030.85 { 

2001 


2000 

i 

2001 

2000 

\ 90.0 

• 

2000.43 

90.0000 

1999.86 

1 90.0000 

1 90.0000 

1999.86 ! 

l 

2000 


2000 

i 

i 

i 

■ 

2000 

2000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT ( GMD) * WHITE SANDS (CYBER) AND SPHERICAL SLAB (VAR I AN) CORRECTION METHODS 


BASED. ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-6G 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION ; .0003307 SCALE HEIGHT; 6631. S 4 TARGET ALTITUDE; SOOO METERS 


INPUT 

DATA 1 

GMD GRAD 

REF SOL i 

CYBER 1 

SPHERICAL SLAB i 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 
DEG 

R MEAS ! 

METER i 

E COR 
DEG 

R COR ! 

METER 

E COR ! 
DEG I 

E COR 
DEG 

R COR : 

METER 1 

NO CORR 1 
METER 1 


GMD H 
METER 


CYB H : 

METER 

VAR H 
METER 

0.0 

! 

298306.60 i 

- , 3799 

i 

298225.86 J 

j 

-.4583 ! 

-.3254 

\ 

298233.94 i 

6972 


4993 


' 

4588 i 

5276 

.S 

229152.64 1 

.2226 

229094.40 { 

.1971 ! 

.2174 

229092.36 \ 

6113 


5002 


4903 I 

4982 

i.O 

179723.87 

.7886 

179679. 7S 

.7799 S 

.7905 

179679. OS l 

5666 


5002 


4976 S 

50 08 

i.S 

144662.69 ! 

1,3327 

144627.87 ! 

1.3300 2 

1.3374 

144627.93 ! 

S42S 


5002 


4996 1 

5013 

2.0 

119495.85 

1.8633 

119467.41 

1.8629 l 

1.8670 

* 119467.74 1 

5288 


5001 

I 

5002 

5009 

2.5 

101023.66 1 

2.3852 

100999.79 l 

2.3858 1 

2.3869 

101000.14 ! 

5205 


5001 

! 

5003 ! 

5004 

3.0 

87104.47 

2.9016 

87083.98 

2.9024 t 

2.9017 

87084.26 J 

5151 


5001 

i 

5003 

5001 

3.5 

76352.98 ! 

3.4140 

76335.07 

3.4150 i 

3.4137 

7633S.27 

5116 


5001 


5003 

5000 

4.0 

67848.80 

3.9239 

67832.92 

3.9248 

3.9245 

67833.04 1 

5092 


S001 

i 

5003 ! 

5001 

4.5 

60982.52 i 

4.4318 

60968.27 

4.4327 i 

4.4348 

60968.33 : 

■ 5074 


5001 


5003 1 

5004 

5.0 

55338.44 

4.9383 

55325.52 

4.9391 

4.9411 

55325.53 \ 

5061 


5001 

< 

i 

5003 : 

50 03 

6.0 

46638.32 

5.9482 

46627.45 

5.9489 1 

5.9497 

46627.41 ! 

5044 


5000 


5002 I 

5002 

7.0 

40268.39 

6.9555 

40259.01 

6.9561 

6.9562 

40258.96 ! 

5033 


5000 

! 

5002 ! 

5001 

8.0 

35416.45 

7.9611 

35408.21 { 

7.9615 ! 

7.9613 

35408.16 i 

5025 


5000 

i 

5002 { 

500 1 

9,0‘ 

31604.31 

8.9654 

31596.96 J 

8.9658 ! 

8.9654 

31596.92 ! 

5020 


5000 

! 

5002 ! 

5000 

10.0 

28533.77 

9.9690 

28527,14 ! 

9.9692 i 

9.9688 

28527.11 ! 

5017 


5000 


5002 i 

5000 

12.0 

23900.03 

11.9743 

23894.48 1 

11.9744 ! 

11.9739 

23894.46 i 

5012 


5000 

! 

5001 J 

5000 

14.0 

20575.95 

13.9781 

20571.17 ! 

13.9781 ! 

13.9776 

20571.16 i 

5009 


5000 

i 

5001 : 

5000 

1 16.0 

18079.69 

15.9811 

1807S.49 ! 

15.9810 I 

! IS. 9805 

18075.48 ! 

5007 


5000 

i 

5001 ! 

5000 

18.0 

16139.36 

17.9833 

16135.61 ! 

17.9832 ! 

17.9828 

16135.61 ! 

5006 


5000 

i 

5001 J 

5000 

I 20.0 

14590 . 17 

19.9852 

14586.78 I 

19.9850 1 

t 19.9846 

14586.78 i 

5005 


5000 

• 

■ 

5001 : 

5000 

25.0 

11817.83 

24,9885 

11815.09 J 

24.9883 1 

24.9879 

11815.08 i 

5003 


5000 

• 

5001 ! 

sooo 

! 30.0 

9993.53 

29.9908 

9991.21 ! 

29.9905 ! 

29.9903 

9991.21 ! 

5003 


5000 

• 

• 

5001 1 

5000 

35.0 

8714. OS 

34.9925 

8712.03 I 

34.9922 ! 

34.9920 

8712.03 J 

5002 


5000 


sooi : 

5000 

! 40.0 

7777.19 

39.9938 

7775.39 ! 

39.9935 ! 

39.9933 

7775.38 i 

5002 


5000 

• 

i 

5001 I 

5000 

45.0 

7070.64 

44.9948 

7069.00 ! 

44.9945 ! 

44.9944 

7069.00 : 

5002 


5000 

i 

• 

5001 J 

5000 

! 50.0 

6527.20 

49.9957 

6525.69 

49.9954 

49,9953 

6525.63 J 

5002 


5000 

• 

• 

sooi : 

5000 

60,0 

5774.28 

59.9971 

5772.94 1 

59.9968 1 

59.9967 

5772.94 1 

5001 


5000 

i 

5001 I 

SOOO 

! 70,0 

5321,92 

69.9982 

5320.69 

69.9980 

69.9979 

5320.68 I 

5001 


5000 

, 

5001 i 

500 0 

80.0 

5078.27 

79.9991 

5077.09 ! 

79.9990 ! 

79.9990 

5077.09 ! 

5001 


5000 

i 

i 

5001 ! 

500 0 

! 90.0 

1 "" 

5001.16 

90.0000 

1 

5000.00 

90.0000 

1 

90.0000 

1 

5000.00 \ 
• 

5001 


5000 

• 

i 

• 

5001 ! 

i 

1 

5000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYDER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0003307 SCALE HEIGHT! 6631. SA TARGET ALTITUDE! 10000 METERS 


: INPUT 

DATA 

1 GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB 1 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

S E MEAS 
I DEG 

R MEAS • 
METER 

S E COR 
i DEG 

R COR 
METER 

E COR 
DEG 

E COR 
DEG 

R COR 1 

METER S 

NO CORR 
METER 


GMD H 
METER 



CYB H 
METER 

• 

• 

• 

• 

VAR H 
METER 

! 0.0 

414831.76 

• 

i -.4823 

414736.25 

-.6374 

-.3709 

! 

414753.81 1 

13476 


9985 


1 

8869 

t 

• 

i 

i 

10791 

: .5 

343319.55 

i .1292 

343247.62 

.0756 

.1666 

343248.89 J 

12224 


10002 



9686 

■ 

i 

10226 

\ i . t) 

287486.54 

1 .7044 

287429,88 

.6861 

.7271 

287430.66 t 

11486 


10002 


1 

9914 

i 

i 

10116 

i i .5 

243847.66 

1 1.2575 

243801.50 

1.2532 

1.2732 

243802.35 \ 

11035 


10002 



9987 

t 

• 

10069 

S 2.0 

209571,06 

{ 1.7962 

209532.45 

1.7980 

1.8065 

209533.28 1 

10748 


10002 



10011 

• 

i 

10039 

1 2.5 

182437.63 

I 2.3254 

182404.63 

2.3298 i 

2.3311 

182405.31 1 

10558 


10002 



10018 

• 

• 

10020 

! 3.0 

160721.61 

I 2.8480 

160692.91 

2.8534 

2.8505 

160693.40 I 

10428 


10001 


1 

10 018 

s 

10008 

! 3.5 

143147.87 

J 3.3658 

143122.55 

3.3715 

3.3670 

143122.84 i 

10337 


10001 • 



10017 

t 

« 

100 04 

; 4.0 

128735.18 

1 3.8801 

128712.56 

3.8858 

3.8824 

128712.69 1 

10271 


10001 



10016 

I 

t 

10006 

i 4.5 

116766.66 

{ 4.3918 

116746.25 

4.3974 i 

4.3974 

116746.27 i 

10222 


10001 


1 

10014 

1 

< 

10012 

! 5.0 

106709.60 

S 4.9016 

106691.02 

' 4.9069 

4.9081 

106690.96 I 

10185 


10001 



10012 

• 

1 

10013 

: 6.o 

90833.09 

i 5.9168 

90817.36 

5.9216 

5.9206 

90817.21 i 

10133 


10001 


' 

10010 

I 

1 

10007 

! 7.0 

78936.99 

1 6.9281 

78923.37 

6.9324 

6.9304 

78923.21 ! 

10100 


10001 



10008 

t 

1 

100 04 

! 8.0 

69733.56 

! 7.9368 

69721.56 I 

! 7.9406 J 

7.9382 

69721.42 ! 

10078 


10000 



10007 

1 

I 

1 00 02 

I ‘ 9.0 

62423.14 

J 8.9437 

62412.41 ‘I 

1 8.9471 I 

8.9446 

62412.31 { 

10063 


10000 • 


! 

10006 

1 

• 

10001 

! 10.0 

56488.11 

J 9.9493 

56478.41 I 

! 9.9524 ! 

9.9498 

56478.34 J 

10051 

i 

10000 


1 

10005 

1 

1 

10001 

: 12.0 * 

47460.07 

1 11.9578 

' 47451.94 : 

! 11.9604 ! 

! 11.9579 

47451'. 89 ' 

10036 

• 

i 

10000 


} 

10004 

1 

i 

100 00 

; 14.0 

40936.50 

\ 13.9640 

40929.49 I 

1 13,9662 ! 

13.9639 

40929.46 S 

10027 

t 

10000 


* 

* 

10003 

i 

l 

1 0000 

! 16.0 

36014.88 

I 15.9687 

36008.72 I 

1 15.9705 ! 

! 15.9685 

36008.70 } 

10021 

• 

t 

10000 


1 

10003 

i 

l 

1 00 0 0 

! 18.0 

32177.39 

1 17.9724 

32171.89 ! 

1 17.9740 J 

17.9721 

32171.87 ! 

10017 

i 

i 

10000 


s 

10003 

l 

l 

10000 

i 20.0 

29106.76 

i 19,9754 

29101.78 ! 

! 19.9767 j 

1 19,9750 

29101,77 { 

10014 

i 

• 

10000 


1 

1 

10003 

l 

4 

1 00 0 0 

i 25.0 

23598.61 

1 24.9809 

23594.58 1 

I 24.9818 I 

24.9805 

23594.57 1 

10009 

i 

i 

10000 



10002 

9 

1 

1 00 0 0 

! 30.0 

19966.15 

! 29.9846 

19962.74 

! 29.9853 ! 

! 29.9842 

19962.74 ! 

10006 

i 

i 

10000 


1 

10002 

l 

4 

1000 0 

J 35.0 

17415.33 

J 34.9873 

17412.36 ! 

! 34.9879 { 

! 34.9869 

17412.35 J 

10005 


10000 


1 

10002 

1 

4 

100 00 

: ' 40.0- 

15546. 14 

I 39.9895 

15543.49 ! 

1 39.9899 ] 

I 39.9891 

15543.48 t 

10004 

i 

t 

10000 


1 

10002 

1 

1 

10000 

i 45. 0 

14135.75 

! 44.9912 

14133.34 ! 

! 44.9915 J 

44.9909 

14133.34 i 

10003 


10000 


! 

10002 

« 

4 

1 0 0 0 0 

! 50.0 

13050.58 

! 49.9926 

13048.35 1 

! 49.9929 ! 

[ 49.9923 

13048.35 1 

10003 

! 

10000 


1 

10002 

1 

4 

10000 

; 60.0 

11546.58 

I 59.9950 

11544.61 1 

! 59.9951 { 

59.9947 

11544.61 1 

10002 

V 

10000 


1 

10002 

4 

1 

10000 

! 70.0 

10642.72 

i 69.9968 

10640.90 

i 69.9969 ! 

[ 69.9967 

10640.90 1 

10002 

i 

1 

10000 


4 

10002 

4 

1 

10000 

l 80.0 

10155.80 

* 79.9985 

10154.07 ! 

i 79.9985 i 

1 79.9984 

10154.07 ! 

10002 

i 

10000 


1 

1 

10002 

1 

1 

10000 

J 90.0 

10001.71 

! 90.0000 
l 

10000.00 

i 90.0000 

i i 

l « 

! 90.0000 

10000.00 1 
1 

10002 

4 

1 

i 

• 

10000 


4 

1 

i 

10002 

• 

1 

1 

1 

1 0 0 0 0 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD ) , WHITE SANDS (CYBER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION: ,0003307 SCALE HEIGHT: 6631.54 TARGET ALTITUDE : 20000 METERS 


tXJ 

I 

a\ 


! 


INPUT 

DATA * 

GMD GRAD 

REF SOL 1 

CYBER 1 

SPHERICAL SLAB l 

ALTITUDE COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 
DEG 

R MEAS 1 

METER ! 

E COR 
DEG 

R COR i 

METER i 

‘ E COR : 
DEG ! 

E COR 
DEG 

R COR : 

METER J 

NO CORR ! 
METER S 

GMD H 
METER 

. 

CYB H 
METER 5 

VAR H 
METER 

0.0 

1 

573683.54 i 

- .5779 

• 

573579.63 ! 

S 

-.8814 

-.4455 

■ 573603.62 1 

\ 

25748 1 

19974 


\ 

16952 J 

21297 

: .s 

500651.97 ! 

.0447 

500572.63 J 

-.0680 1 

.0973 

500575.58 1 

23973 \ 

20002 


19027 t 

20460 

s i .o 

440432.58 

.6286 

440369.16 

.5832 

* .6617 

440370.83 i 

22852 J 

20002 • 


19659 1 

20256 

: i.5 

390259.46 ! 

1.1891 

390207.08 l 

1.1722 1 

1.2115 

390208.38 1 

22117 S 

20002 


19892 J 

20154 

\ 2.0 

348138.04 

1.7342 

348093.69 

1.7307 

1.7487 

348094.73 ! 

21614 i 

20002 

: 

19984 ! 

20090 

: 2.5 

312596.55 

2.2690 

312558,26 ! 

2.2721 

2 • 2772 

312559.01 ! 

21260 ! 

20002 


20022 i 

20046 

i 3.0 

282451.67 

2.7965 

282418.09 

2.8029 

2.8003 

282418.53 ! 

21002 : 

20000 

i 

20035 : 

20019 

: 3.5 

256791.90 J 

3.3186 

256762,07 ! 

3.3266 

3.3206 

256762.25 \ 

20812 ! 

20000 


• 20039 J 

20009 

! 4.0 

234814.60 

3.8367 

234787.81 

* 3.8454 

3,8399 

234787.78 S 

20669 \ 

20000 

i 

• 

20039 : 

20013 

I 4,5 

215882.48 

4,3517 

215858.19 { 

4.3607 

4.3593 

21S8SB.03 1 

20558 1 

20000 


20037 { 

20029 

; s.o 

199475.77 

4.8644 

199453.58 

4.8733 

‘ 4.8733 

199453.32 i 

20472 ! 

20000 

i 

20034 J 

20031 

; 6.o 

172626.25 

5.0845 

172607.36 S 

S . 8929 

5.8899 

172607.02 ! 

20348 J 

20000 

i 

20028 : 

20016 

: 7.o 

151744.07 

6.8996 

151727.65 

6.9074 

6.9030 

151727,33 ! 

20266 ! 

20000 

! 

20023 : 

20009 

; 8 . o 

135149.28 J 

7.9114 

135134.77 1 

7.9186 

7.9137 

135134.51 i 

20209 2 

20000 

i 

20019 1 

20006 

: ?.o 

121705 . 50 I 

8.9208 

121692.51 1 

8.9274 

1 8.9224 

121692.31 i 

20168 2 

20000 

■ 

20016 ! 

20004 

» 10.0 

110628.54 ! 

9.9235 

110616.78 I 

9.9346 

9.9296 

110616.63 ! 

20138 1 

20000 

• 

20014 1 

20002 

! 12.0 

93518.36 I 

11.9403 

93508.48 ! 

11,9455 

I 11.9408 

93508.38 i 

20097 J 

20000 

i 

20011 { 

20001 

I 14.0 

80975.81 ! 

13.9489 

80967.28 1 

13.9534 

13.9491 

80967.22 I 

20072 J 

20000 

J 

20009 S 

20001 

! 16.0 

71424.22 1 

15 ; 9555 

71416.72 I 

15.9594 

! 15.9555 

71416.68 i 

20056 2 

20000 

1 

20007 1 

20000 

i 18.0 

63928.78 J 

17.9607 

63922.07 1 

17.9641 

17.9606 

63922.05 i 

20044 2 

20000 

■ 

• 

20006 ! 

20 000 

i 20.0 

57903.68 J 

19.9649 

57897.61 I 

19.9679 

! 19.9648 

57897.59 \ 

20036 2 

20000 

i 

20005 ! 

20000 

J 25.0 

47041.46 ! 

24.9726 

47036.54 ! 

24.9749 

24.9724 

47036.53 i 

20023 2 

20000 

• 

i 

20004 { 

20000 

S 30.0 

39844.96 ! 

29,9779 

39840.80 ! 

29.9797 

! 29.9776 

39840.79 ! 

20016 J 

20000 

i 

20003 1 

2000 0 

; 35.0 

34777.96 ! 

34.9818 

34774.33 ! 

34.9833 

1 34.9815 

34774.33 ! 

20011 2 

20000 

« 

■ 

20003 i 

20000 

: 40.0 

31050.83 ! 

39.9848 

31055.59 ! 

39.9860 

1 39.9846 

3105S.59 J 

20008 2 

20000 

J 

20003 1 

20000 

I 45.0 

28249.49 1 

44.9873 

28246. S4 { 

44.9883 

! 44.9871 

28246.54 1 

20007 2 

20000 

■ 

20003 ! 

20 000 

; so.o 

26086.33 : 

49,9893 

26083.61 ! 

49.9902 

! 49.9891 

26083.61 ! 

20005 2 

20000 

i 

20002 : 

20000 

: 6o.o 

23086.13 I 

59.9927 

23083.72 { 

59.9932 

I 59.9925 

23083.72 i 

20004 2 

20000 

* 

20002 J 

20000 

{ 70.0 

21282.00 : 

69.9954 

21279,78 ! 

69.9957 

1 69.9953 

21279.78 \ 

20003 2 

20000 

• 

20002 ! 

20000 

I 80.0 

20309.81 I 

79,9978 

20307,69 i 

79.9979 

I 79,9977 

20307.69 ! 

20002 2 

20000 

i 

20002 J 

20000 

i 90.0 

20002.09 

90,0000 

20000.01 : 

90,0000 

1 90.0000 

20000,00 ! 

20002 2 

20000 

1 

20002 i 

20000 


i 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB < VAR I AN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0003307 SCALE HEIGHT! 6631. S4 TARGET ALTITUDE! S0000 METERS 


INPUT 

DATA ! 

! GMD GRAD 

REF SOL ! 

! CYBER * ! 

SPHERICAL SLAB 1 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

1 

METHOD 

E ME AS 

R MEAS ! 

! E COR 

R COR I 

1 E COR I 

E COR. 

R COR I 

NO CORR 

j 


GMD H ! 

! 

CYB H 

! 

VAR H 

DEG 

METER I 

DEG 

METER ! 

DEG ! 

DEG 

METER » 

METER 

i 


METER 1 

t 

METER 

: 

METER 

0 . 0 

878563.13 

-.6747 

i 

878455.62 ! 

1 

-1.3499 1 

1 -.5537 

1 

878482.82 S 

60225 

i 

» 

i 

49954 

i 

i 

39689 

• 

• 

i 

51799 

.S 

804288.45 

-.0358 

804206.57 I 

-.2875 ! 

-.0177 

804208 . 49 i 

57470 

i 

» 


50001 : 

1 

46505 

\ 

50254 

i .0 

740413 . 56 1 

.5595 

740348.14 i 

.4446 S 

.5687 

740348.84 ! 

55655 

i 

t 

S0002 

i 

i 

48536 

: 

50120 

1.5 

684451.21 1 

1.1285 

684397.14 

• 1.0714 1 

1.1342 

684397.83 i 

54410 


• 

50002 

i 

49332 

■ 

* 

50070 

2.0 

634797.11 

1.6803 

634751.28 ! 

1.6514 ; 

1.6827 

634751.90 S 

53519 

i 

1 

50002 

i 

> 

49690 

• 

i 

50028 

2.5 

590419.43 

2.2205 

590379.81 

2.2065 ! 

2.2193 

590380.23 ! 

52862 


! 

50002 

l 

49865 

! 

49990 

3.0 

550557.68 

2.7523 

550S22.88 ! 

2.7468 { 

2.7487 

550523.06 ! 

52361 



49998 

i 

i 

49950 

i 

. 

49964 

3.5 

514696.18 

3.2781 

514665.22 i 

3.277S ! 

3.2743 

• 514665.16 ! 

51977 



49999 

\ 

49998 

. 

• 

49965 

4.0 

482335.36 ! 

3.7994 

482307.52 1 

3.8017 { 

3.7986 

482307.28 I 

51674 



49999 

• 

• 

50022 

* 

« 

49992 

4,5 

4S3077 . 14 I 

4.3172 

453051.87 ! 

4.3213 1 

1 4.* 3228 

453051.52 * 

51433 



49999’ 

i 

50035 

• 

• 

50044 

S.O 

426575.04 \ 

4.8323 

426551.93 i 

4.8374 ! 

4.8403 

426551.50 i 

51237 



50000 

! 

SO 041 

i 

i 

50058 

6.0 

380655.31 ! 

• S , 8563 

380635.60 ! 

5.8624 

1 S . 8611 

-380635.12 1 

50945 



50000 


50043 

1 

1 

50031 

7.0 

342537.03 ! 

6.8747 

342519.88 I 

6.8809 

6.8777 

342519.45 i 

50741 



50000 

! 

50039 

i 

50018 

8.0 

310625.15 I 

7.8890 

310609.98 ! 

7.8951 

I 7.8910 

310609.63 . 

50594 



50000 

. 

• 

50035 

i 

i 

5001 1 

9.0 

283675.10 1 

8,9006 

283661.50 I 

8.9064 

8.9020 

283661.23 *. 

50485 



soooo 


50031 

1 

1 

50007 

10. 0 

260717.55 ! 

9.9101 

260705.24 ! 

9.9155 

9.9110 

26070S , 03 1 

50403 



50000 

i 

50027 

i 

i 

50004 

12.0 

223921.63 ! 

11,9247 

223911.27 1 

11.9295 

11.9252 

223911.15 i 

50288 



50000 



t 

i 

50002 

14.0 

195943.95 ! 

13.9354 

195935.01 ! 

13.9397 

13:9357 

195934.93 J 

50215 



50000 

i 

50017 

* 

i 

5000 1 

16,0 

174095.46 ! 

15.9436 

174087.59 I 

IS. 9474 

15.9438 

174087.53 S 

50166 



50000 

i 

50014 

. 

* 

50001 

18.0 

156641.05 i 

17.9501 

156634.01 

17.9535 

17.9502 

156633.97 J 

50131 



50000 

* 

■ 

50011 

i 

i 

5000 0 

20.0 

142425.79 ! 

19.9554 

142419.42 ! 

19.9585 

19.9554 

142419.39 J 

50106 


\ 

50000 

i 

i 

50010 

i 

i 

50000 

25.0 

116417.69 ! 

24.9651 

116412.53 

■ 24.9675 

24.9650 

116412.51 1 

50066 


1 

50000 

i 

■ 

50007 

t 

t 

SOOOO 

30.0 

98946.19 1 

29.9718 

98941.82 ! 

29.9737 

29.9717 

98941.81 1 

50044 



50000 

• 

50005 

t 

i 

SOOOO 

35.0 

86544.67 1 

34,9767 

86540.86 

34.9783 

34.9766 

•86540.85 J 

50031 



50000 

i 

50004 

• 

i 

50000 

40.0 

77395.44 i 

39.9806 

77392,04 1 

39.9819 

39.980S 

77392.03 ! 

50022 



SOOOO 

i 

50004 

i 

i 

5000 0 

45.0 

70460.73 ! 

44.9837 

70457.63 I 

44.9848 

44.9836 

70457.63 S 

50016 



50000 

• 

i 

50003 

i 

i 

50000 

50.0 

65108.41 1 

49.9863 

65105.55 1 

49,9873 

49.9862 

65105.55 I 

50012 



50000 


SO 0 03 

« 

i 

SOOOO 

60.0 

57668.35 ! 

59.9906 

57665.82 1 

59.9912 

59.9905 

5766S.82 ! 

50007 



50000 

t 

50003 

t 

i 

50000 • 

70.0 

53185.84 i 

69.9941 

53183.51 1 

69.9945 

69.9940 

53183.51 1 

50004 



50000 

t 

i 

50002 

1 

50000 

80 . 0 

50767.87 

79,9971 

50765.65 

79.9973 

79.9971 

50765.64 5 

50003 



50000 

i 

50002 

i 

i 

50000 

90.0 

50002.19 ! 

90.0000 

50000.00 J 

90.0000 

i 

90 .0000 

50000.00 i 
1 

50002 



50000 

t 

i 

50002 

! 

: 

50000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 


COMPARISON OF GRADIENT (GMD ) , WHITE SANDS (CYDER) AND SPHERICAL SLAD ( VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION i .0003307 SCALE HEIGHT! 6631.54 TARGET ALTITUDE! 100000 METERS 


l INPUT 

DATA S 

GMD GRAD 

REF SOL 1 

CYDER 1 

SPHERICAL SLAB ! 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

J E MEAS 

R MEAS 

E COR 

R COR ! 

E COR ! 

E COR 

R COR : 

NO CORR 


GMD H 


CYB H J 

VAR H 

! DEG 

METER l 

DEG 

METER i 

DEG I 

DEG 

METER ! 

METER 


METER 


METER 

METER 

1 0.0 

t 

1217288.16 

-.7261 

, 

1217179.38 S 

-1.8703 \ 

‘ -.6148 

1217207.77 1 

115122 

i 

99934 


75979 S 

102265 

l .5 

1142417.30 

-.0758 

1142334.76 ! 

-.4706 t 

-.0830 

1142337.33 1 

111309 


100001 


92261 S 

99861 

» 1.0 ■ 

1076774.52 

. S272 

1076708.72 J 

.3446 ! 

.5190 

1076708.62 i 

108757 


100003 


96635 1 

99851 

J i.S 

1017929.38 

1.1015 

1017875.08 

■ 1.0053 1 

1.0952 

1017875.31 J 

106978 

• 

• 

100003 


98331 ! 

99893 

I 2.0 

964367 . 07 

1.6572 

964321.09 1 

i .6029 ! 

1.6510 

964321.44 1 

10S686 


100003 


99111 J 

99900 

J 2.5 

915169 . 95 

2.2004 

915130.22 

2 . 1685 1 

2.1928 

915130.48 1 

104716 

, 

• 

100002 

1 

99508 J 

99884 

l 3.0 

869704.42 

2.7345 

869669.55 J 

2.7155 5 

2.7260 

869669.61 ! 

103964 

: 

99997 


99720 J 

99870 

1 3.5 

827598.81 

3.2622 

B27S67 .80 ! 

3.2510 ! 

3.2547 

827567.66 J 

103378 

, 

99998 


99844 j 

99891 

J 4.0 

788486.90 

3.7850 

788459.02 ! 

3.7787 J 

3.7815 

788458.73 ! 

102909 


99998 


99918 ! 

99951 

! 4.5 

752100.64 

4.3041 

752075.34 

4,3009 \ 

4.3081 

752074.95 J 

102528 

i 

99999 


99963 I 

100051 

I 5.0 

718212. 5S 

4.8202 

718189.42 1 

4.8191 : 

4.8272 

718188.95 S 

102214 

i 

99999 


99990 1 

100086 

l 6.0 

657153.64 

5.8459 

657133 . 92 

5.8472 J 

5.8500 

657133.41 1 

101734 

i 

100000 


100017 ! 

100045 

S 7.0 

603937.46 

6.8655 

603920.30 

6.8678 S 

6.8681 

603919.85 S 

101389 

i 

100000 


100027 S 

100026 

\ 8.0 

557415.60 

! 7,8809 

557400.42 

7.8837 1 

7.8826 

557400.06 \ 

101133 

i 

100000 


100029 S 

100015 l 

i 9.0 

516611.36 

8.8933 

516597.76 

8.8962 1 

8.8944 

516597.48 • 

100939 


100000 


100029 ! 

100010 

\ i 0 . 0 

480692.61 

I 9.9034 

480680.29 

• 9.9064 ! 

9.9042 

■ 480680.08 \ 

100789 


100000 


100027 ; 

100006 

: i 2 .o 

420792.89 

1 11.9191 

420782.53 

11.9219 ! 

11.9195 

420782.40 ! 

100575 


100000 


100023 J 

100003 

! 14.0 

373274,02 

[ 13.9306 

373265.08 

13.9331 1 

13.9308 

373264.99 \ 

100435 


100000 


100019 ! 

100001 

16.0 

334973.94 

i 15.9394 

334966.07 

15.9417 ! 

15.9395 

334966.01 *, 

100338 


100000 


100016 1 

10000 1 

! 18.0 

303639.19 

1 17.9463 

303632.15 

17.9485 ! 

17.9464 

303632.11 ! 

100269 

i 

100000* 


100013 J 

100000 

I 20.0 

277650 .34 

! 19.9520 

277643.97 

19.9539 ! 

19.9520 

277643.94 1 

100218 


100000 


10 0 011 { 

100000 

! 25.0 

229070.13 

! 24.9624 

229064.97 

24.9640 ! 

24.9624 

’ 229064.95 ! 

100137 


100000 


10000Q ! 

10000 0 

! 30.0 

195745.38 

! 29.9696 

195741.01 

29.9709 J 

29.9695 

195741.00 J 

100091 


100000 


100006 : 

100 000 

! 35.0 

171789.66 

! 34.9749 

171785.85 

34.9759 I 

34.9748 

171785.84 i 

100063 


100000 


100005 i 

100000 

! 40.0 

153971.50 ! 

1 39.9790 

153968.10 

39.9799 i 

39.9790 

153968.09 i 

100045 


100000 


100004 J 

100000 

I 45.0 

140391.31 

1 44 . 9824 

140388.21 

44.9831 ! 

44.9324 

140388.21 ! 

100032 


100000 


ioooo4 : 

1000 0 0 

! 50.0 

129869.19 

! 49.9852 

129866.33 

49.9859 i 

49.9852 

129866.33 ! 

100023 


100000 


100003 * 

10 0000 

: 6o.o 

115188.61 • 

i 59.9898 

115186.08 

59.9903 ! 

59.9898 

115186.08 ! 

100012 


100000 


100003 i 

1000 0 0 

: 70.0 

106315.79 

! 69.9936 

106313.46 

69.9939 1 

69.9936 

106313.46 i 

100006 


100000 


iooooa : 

100000 

; 80.0 

101521.49 

! 79.9969 

101519.27 

79.9970 ! 

79,9969 

101S19.26 : 

100003 


100000 


100002 ! 

100000 

: 9o.o 

i 

100002.19 

! 90.0000 
! 

100000.00 

90.0000 ! 

90.0 000 

100000.00 : 
1 
1 

100002 


100000 


100002 1 
t 
* 

1 00000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



APPENDIX C 

TABULATED REFRACTION COMPARISON DATA 
FOR SEA-LEVEL MODULUS OF REFRACTION OF 0.0003860 


C-l 



COMPARISON OF GRADIENT ( GhD ) , WHITE SANDS <CYBER> AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0003860 SCALE HEIGHT! S610.V3 TARGET ALTITUDE! 1000 METERS 


INPUT 

DATA t 

GMD GRAD 

REF SOL 

CYBER 5 

SPHERICAL SLAB 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 
DEG 

R MEAS ! 

METER 1 

E COR 
DEG 

R COR 
METER J 

E COR 1 
DEG i 

E COR 
DEG 

R COR 
METER \ 

NO CORR 
METER 



GMD H 
METER 


CYB H : 

METER i 

VAR H 
METER 

0.0 

148875.73 1 

-.2836 

148820.91 ! 

******* J 

-.2872 

148815,71 ! 

1737 



1000 


1 

ft 

******* J 

990 

.5 

80619.94 1 

.3502 

80591.10 : 

******* s 

.3446 

80588.01 ! 

1213 



1002 


ft 

^ nh ^ ♦ T* n* T 1 

994 

i.O 

50596.50 

.9069 

50578.53 

******* : 

.9073 

50577,81 l 

1084 



1001 


******* I 

1001 

i.S 

35950.45 I 

1.4344 

35937.71 ! 

******* i 

1.4355 

35937. S5 1 

1042 



1001 


******* ! 

1001 

2.0 

27658.47 

1.9498 

27648.68 

******* : 

1.9502 

27648.66 ! 

1025 



1001 


******* } 

1001 

2 . 5 

22406.06 S 

2.4596 

22398.13 

******* ; 

2.4591 

22398.15 i 

1017 



1000 


******* i 

1000 

3.0 

18804.16 

2.9663 

18797.51 

******* i 

2.9653 

18797.53 ! 

1012 


\ 

1000 


******* J 

1000 

3.5 

16189.44 1 

3.4712 

16183.72 

******* : 

3.4700 

16183.73 I 

1009 



1000 


******* : 

1000 

4.0 

14208.17 

3.9749 

14203.15 

******* | 

3.9741 

14203.15 ! 

1007 


, 

i 

1000 


******* : 

1000 

4.5 

12656,65 

4.4778 

12652.18 

******* j 

4.4778 

12652.18 l 

1006 



1000 


******* ; 

.1000 

i s.o 

11409.57 

4.9801 

11405.54 

******* 

4.9792 

11405.53- J 

1005 


\ 

1000 


******* \ 

1000 

: 6.o 

9530.61 

5.9836 

9527.24 

******* : 

5.9826 

9527.24 ! 

1003 


\ 

1000 


i 

ft 

1000 

! 7.0* 

8183. SO 

6.9861 

8180.61 

******* j 

6.9850 

8180.60 1 

1002 


\ 

1000 


2|C ^ ft 

1000 

: s.o 

7171.17 1 

7.9880 

7168.64 S 

1 

T 'r ^ ^ ^ 1 

7.9869 

7168.63 i 

1002 


i 

1000 


******* ; 

1000 

1 9.0 

6383,02 1 

! 8.9895 

6380,76 ! 

******* J 

8,9883 

6380.76* i 

1002 


« 

• 

1000 


******* s 

1000 

I 10,0 

5752.30 J 

9.9907 

5750.27 J 

v 

^ ^ ^ T T | 

9.9895 

5750.26 ! 

1001 


1 

1000 


******* ; 

1000 

! 12.0 

4806.54 1 

1 11.9925 

4804.84 1 

******* J 

11.9913 

4804.84 * ! 

1001 


• 

i 

1000 


******* : 

1000 

! 14.0 

4131.97 J 

13.9937 

4130,51 1 

******* : 

13.9926 

4130.51 ! 

1001 


■ 

i 

1000 


******* 5 

1000 

1 16.0 

3627,21 1 

I 15.9947 

3625.93 

******* J 

i 15.9935 

3625.93 I 

1001 


i 

■ 

1000 


* 

1000 

: i8.o 

3235.80 1 

17.9954 

3234.66 i 

******* ; 

17.9943 

3234.66 I 

1001 


1 

1000 


******* | 

1000 

: 20.0 

2923.82 1 

1 19.9960 

2922.79 

******* i 

[ 19.9949 

2922.79 ! 

1001 


i 

• 

1000 


******* : 

1000 

i 2S.0 

2366.54 ! 

I 24.9971 

2365.71 ! 

******* j 

24.9960 

2365.70 I 

1001 


i 

i 

1000 


******* : 

1000 

1 30.0 

2000.43 ! 

I 29.9978 

1999.72 

! ******* 

29,9968 

1999.72 1 

1000 


■ 

i 

1000 


******* \ 

1000 

i 35.0 

1743.90 1 

! 34.9983 

1743.29 ! 

1 ******* 

34.9973 

1743.28 1 

1000 


1 

1000 


******* i 

1000 

i 40.0 

1556.17 ; 

1 39.9987 

1555.62 

} ******* 

39.9978 

1555.62 i 

1000 


i 

1000 


******* J 

1000 

J 45.0 

1414.65 ! 

1 44.9990 

1414.15 

! ******* 

44.9981 

1414.15 1 

1000 


1 

1000 


******* : 

1000 

1 50.0 

1305.83 ; 

! 49.9992 

1305.37 

1 ******* 

49.9984 

1305.37 S 

1000 


« 

1000 


******* ' ! 

1000 

; 60.0 

1155.09 I 

I 59.9995 

1154.68 

1 ******* 

S9.9989 

1154.68 i 

1000 


fl 

1000 


******* : 

1000 

: 70.0 

1064.55 

! 69.9997 

1064,18 

i ******* 

69.9993 

1064.17 1 

1000 


1 

1000 


******* ft 

^ ^ •!» 'v ^ ^ ^ I 

1000 

J 80.0 

1015.78 I 

1 79.9999 

1015.42 ! 

i ******* 

79.9997 

1015.42 « 

1000 


i 

• 

1000 


******* ] 

1000 

! 90.0 

1000.35 

! 90.0000 

1 

1 

1000.00 

1 ******* 
i 

90.0000 

• 1000.00 1 
1 
ft 

1000 


1 

i 

1000 


******* « 

1000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OK GRADIENT (GMD) , WHITE SANDS (CYDER > AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION: .0003860 SCALE HEIGHT: 5610.93 TARGET ALTITUDE: 2000 METERS 


• ■ r ■■ ■■ ■" - 

INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB * 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 
DEG 

R MEAS 
METER 

E COR 
DEG 

R COR 
METER 

E COR 
DEG 

E COR 
DEG 

R COR \ 

METER ! 

NO CORR 
METER 


GMD H 
METER 

CYB H J 

METER 

VAR H 
METER 

0.0 

208511.69 

-.3872 

208438.26 

******* 

* -.3852 

• 

208441.17 1 

3407 


1997 

1 

******* | 

2004 

.5 

133433.31 

.2611 

133388.53 

******* 

.2474 

133384.00 ! 

2560 


2002 

******* ! 

1970 

i . 0 

91777.42 

.8382 

91747.16 

******* 

,8393 

91745.81 I 

2262 


■ 2002 

******* ; 

2003 

i.S 

68029.26 

i .3810 

68006.99 

******* 

i .3843 

68006.69 ! 

2143 


2001 

******* : 

2005 

2.0 

53435.22 

- 1.9070 

53417.78 

******* 

1.9091 

53417.79 1 

2088 


2001 

******* j 

2003 

2.5 

43776.40 

2.4242 

43762.14 

******* 

2.4247 

43762.21 ! 

2059 


2001 

******* j 

2001 

3.0 

36983. OS 

2.9362 

36971.02 

******* 

2.9356 

36971.08 ! 

2042 


2001 

******* ! 

2000 

3 , S 

31974.82 

3.4450 

31964.42 

******* 

3. 4441 

31964.46 { 

2032 


2001 

******* { 

2000 

4.0 

28140.97 

3.9518 

28131.82 

******* 

3.9515 

28131.84 I 

2025 


2000 

. ******* 1 

200 0 

4,5 

25117.57 

4.4571 

25109,41 

******* 

4.4584 

25109.41 1 

2020 


2000 

******* ; 

200 1 

5.0 

22675.22 

4.961S 

22667.85 

******* 

4.9615 

22667.84 ! 

2016 


2000 

******* } 

2000 

6.0 

18976.96 

5.9680 

18970.80 

******* 

5.9676 

18970.78 i 

2012 


2000 

******* s 

2000 

7. 0 

16313.56 

6.9727 

16308.26 

******* 

6.9720 

16308.24 S 

2009 


2000 

******* } 

2000 

8.0 

14306. 3S 

7.9762 

14301.70 

\ ******* 

7.9754 

14301.69 1 

2007 


2000 

I ******* ! 

2000 

9,0 

12740.69 

1 8.9790 

12736.55 

| ******* 

8.9781 

12736.54 1 

2005 


2000 

1 ******* } 

2000 

10. 0 

11486.07 

9.9813 

11482,34 

j ******* 

9,9803 

11482.33 ! 

20 05 


2000 

: ******* : 

2000 

12.0 

9602.32 

! 11,9846 

9599,20 

i 

I ^ o' ^ ^ 

11.9836 

9599.20 ; 

2003 


2000 

i ******* § 

2000 

14.0 

8257.14 

13.9870 

8254.46 

******* 

13,9860 

8254.46 ! 

2003 


2000 

j ******* j 

2000 

16.0 

7249.85 

! 15.9888 

7247.50 

! ******* 

15,9878 

7247.49 S 

2002 


2000 

l ******* ; 

2000 

18.0 

6468.39 

17.9903 

6466.29 

\ ******* 

17.9892 

6466.29 \ 

2002 


2000 

| ******* ! 

2000 

20 . 0 

5845.29 

! 19.9914 

• 5843.39 

| ******* 

19.9904 

5843.39 * 

2002 


2000 

******* ! 

2000 

25.0 

4731,86 

24.9935 

4730,32 

; ******* { 

24.9925 

4730.32 ! 

2001 

\ 

2000 

4 s ♦ ^ 4* 4* ^ v 1 

2000 

30 . 0 

4000,15 

\ 29.9949 

3998.85 

! ******** | 

29.9939 

3998.85 l 

2001 

1 

2000 

1 

• 2000 

35.0 

3487.35 

34.9959 

3486,22 

! ******* J 

34.9950 

3486.22 ! 

2001 


2000 I 

******* ; 

2000 

40 . 0 

3112.04 

1 39,9967 

3111.03 

j ******* ! 

39.9958 

3111.03 J 

2001 

i 

• 

2000 

******* ! 

2000 

4S.0 

2829,08 

44.9973 

2828.16 

I ******* ; 

44.9965 

2828.16 I 

2001 


2000 

1 

4“ 4 4 4 4 4 4 1 

2000 

50 . 0 

2611 ,49 

1 49.9978 

2610.64 

| ******* | 

49.9971 

2610.64 i 

2001 

• 

2000 

^ ^ 1 

4 4 4 4 4 4 4 1 

2000 

60.0 

2310.08 

59.9985 

2309,33 

******* 1 

59.9980 

2309.33 ! 

2001 


2000 

******* { 

2000 

70 . 0 

2129.02 

! 69.9991 

2128.33 

i ******* : 

69.9987 

2128.33 ) 

2001 

1 

2000 

******* ! 

2000 

80.0 

2031 .51 

79.9996 

2030.85 

! ******* ! 

79.9994 

2030.85 S 

2001 

1 

i 

2000 

******* J 

2000 

90 . 0 

2000.65 

: 90.0000 

2000.00 

{ ******* 
i i 

i > 

90.0000 

2000.00 \ 
* 

2001 

i 

« 

2000 

******* | 
I 

200 0 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT < GMD > , WHITE SANDS (CYDER) AND SPHERICAL SLAB ( VAR I AN ) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 75-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION: .0003G60 SCALE HEIGHT: 5610.93 TARGET ALTITUDE: 5000 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYDER 

SPHERICAL SLAB 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD } 

E MEAS 

R MEAS 

E COR 

R COR 

E COR 

E COR 

R COR I 

NO CORR 


GMD H ! 

CYB H ! 

VAR H 

DEG 

METER 

DEG 

METER 1 

DEG 

DEG 

METER J 

METER 


METER ! 

METER 1 

METER 

0.0 

321912.52 

-.5568 

321812.10 

******* 

-.4899 

i 

321831.83 ! 

8118 


i 

4989 

******* 

5365 

.5 

240923.86 

.1089 

240855.00 

******* 

. 0924 

240851.95 ! 

6649 


5003 ! 

******* 

4934 

• 1.0 

185873.89 

.7085 

185823.12 

******* 

.7068 

185822.02 J 

5950 


5003 

******* 

4997 

i.S 

148057.21 

1.2723 

148017.75 

******* 

1.2767 

148017.74 \ 

5592 


5002 1 

******* 

5013 

2.0 

121482.77 

• 1.8154 

121450.84 

******* 

1.8193 

121451.18 ! 

5394 


5002 

******* 

5010 

2.5 

102254.56 

2.3458 

102227.91 

******* 

2.3474 

102228.26 1 

5278 


5002 

******* 

50 04 

3.0 

87905.49 

2.8682 

87882,70 

******* 

2.8679 

87882.95 1 

5204 


5001 

******* 

50 0 1 

3,5 

76898.74 

3.3851 

7 6878.87 

******* 

3.3845 

76879.01 ! 

5156 


5001 

******* 

5000 

4.0 

68234.87 

3.8984 

68217.28 

******* 

3.8991 

68217.34 ! 

5123 


5001 

******* 

5002 

4,5 

61264.47 

4.4091 

61248.71 

******* 

4.4130 

61248.70 1 

5099 


5001 

******* 

50 05 

1 S.O 

55550 . 01 

4.9178 

55535.74 

£ ^ ^ £££ ^ 

'4.9214 

55535.68 ! 

5081 

I 

5001 

******* 

5004 

! 6,0 

46765.66 

5.9311 

46753,67 

******* 

5.9329 

46753.5 7 ! 

5058 


5001 

******* 

50 02 

1 7.0 

40350 . 59 

6.9409 

40340.25 

******* 

6.9417 

40340.17 S 

5043 

• 

i 

5001 

******* 

500 1 

i 8.0 

35472,45 

7.9483 

35463,37 

; ******* 

7.9485 

35463.30 ! 

5033 

i 

5000 

******* 

5001 

! 9.0 

31644. 13 

8,9541 

31636.03 


8.9540 

31635.98 1 

5027 

i 

5000 

******* 

50 0 0 

I 10.0 

28563,07 

9,9588 

28555.76 

******* 

9,9585 

28555.73 ! 

5022 

• 

5000 

******* 

50 00 

i 12,0 

23917.22 

11,9659 

23911.11 

! ******* 

11.9653 

23911.08 ! 

5016 

i 

5000 

| ******* 

50 00 

: i4.o 

20586.88 

13.9710 

20581.62 

******* 

13.9703 

20581.60 ! 

5012 

• 

• 

5000 

i ******* 

50 00 

1 16,0 

18087.07 

15.9749 

18082,45 

i ******* 

15,9741 

18082.44 ! 

5009 

• 

i 

5000 

s ******* 

5000 

: i8.o 

16144.58 

17.9779 

16140.46 

I ******* 

17,9771 

16140.45 1 

5007 

1 

5000 

! ******* 

5000 

i 20,0 

14594.01 

19,9803 

14590,28 

J ******* 

19.9796 

14590.28 1 

5006 

1 

1 

5000 

1 ******* 

50 0 0 

! 25.0 

11819.83 

24.9848 

11816.81 

i ******* 

24,9840 

11816.81 1 

5004 

1 

5000 

[ ******* 

50 00 

! 30.0 

9994.72 

29.9878 

9992.17 

: ******* 

29.9871 

9992.17 ! 

5003 

. 

5000 

1 ******* 

500 0 

1 35.0 

8714.82 

34,9901 

8712.59 

; ******* 

34.9893 

8712.59 i 

5003 

i 

5000 

; ******* 

5000 

! 40.0 

7777.72 

39.9918 

7775.73 

s ******* 

39.9911 

7775.73 ! 

5002 

• 

5000 

! ******* 

50 00 

! 45,0 

7071 . 03 

44.9932 

7069.23 

I ^ ^ ^ ^ 

1 44.9925 

7069,22 ! 

5002 

i 

5000 

! ******* 

5000 

: 50.0 

6527.50 

49.9943 

6525.83 

: ******* 

1 49,9937 

6525.83 ! 

5002 

• 

5000 

i ******* 

50 00 

J 60.0 

5774.47 

59.9961 

5773,00 

i ******* 

! 59.9957 

5773.00 ! 

5001 

i 

i 

5000 

! ******* 

50 0 0 

! 70,0 

5322.06 

69,9976 

5320,70 

i ******* 

\ 69.9973 

5320.70 i 

5001 

J 

5000 

1 ******* 

5000 

! 80.0 

5078.39 

79.9988 

5077.10 

9 

! 79.9987 

5077.09 J 

5001 

1 

1 

5000 

J ******* 

50 0 0 

J 90.0 

5001.28 

90 .0000 

5000.00 

i ******* 

i 

• 

5 90.0000 

5000.00 1 

9 

5001 

1 

1 

5000 

J ******* 

1 

5000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OK GRADIENT ( GMD > , UHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0003860 SCALE HEIGHT! 5610.93 TARGET ALTITUDE! 10000 METERS 


, ! INPUT 

DATA 

GMD GRAD 

REF SOL 

CYDER 

SPHERICAL SLAB i 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

I E MEAS 
I DEG 

i — 

R NEAS 
METER 

E COR 
DEG 

R COR 
METER 

E COR 
DEG 

E COR 
DEG 

R COR 1 

METER i 

NO CORR 
METER 

GMD H 
METER 

CYB H 
METER 

UAR H 
METER 

! 

; o.o 

442467.95 

-.6935 

442351.10 

******* 

-.5816 

i 

442383.70 1 

15329 

9977 

******* 

10842 

; .5 

358881.42 

-.0129 

358798.18 

******* 

-.0020 

358799.1 7 I 

13215 

10004 

******* 

10072 

• { 1.0 

296788.75 

.6003 

296725.16 

******* 

.6091 

296725.85’ l 

12073 

10003 

******* 

10049 

! i . 5 

249668.60 

1.1767 

249617.75 

******* 

1.1857 

249618.70 \ 

11412 

10003 

******* 

10042 

J 2.0 

213355.14 

1.7309 

213313.10 


i . 7374 

213314.03 1 

11005 

10002 

******* 

10027 

: 2.5 

184983.95 

2.2709 

184948.31 

******* 

2.2738 

184949.01 ! 

10742 

10002 

******* 

10012 

J 3.0 

162487.42 

2.8013 

162456.59 

******* 

2.8015 

162457.02 ! 

10565 

10001 

******* 

10001 

; 3.5 

144410 .59 

3.3251 

144383.49 

'k kk kk k k 

3.3246 

144383.68 I 

10442 

10001 

******* 

ioooo 

J 4.0 

129662.00 

3.8441 

129637.86 

******* 

3.8457 

129637.87 i 

10354 

10001 

******* 

1 00 04 

S 4.5 

117463.01 

4,3596 

117441,27 

******* 

4.3660 

117441.17 1 

10289 

10001 

******* 

10014 

O ! 5.0 - 

107243.77 

4.8724 

107224.01 

******* 

4.8800 

107223.83 i 

10240 

10001 ■ 

******* 

1001S 

t ! 6.0 

91164.64 

5.8924 

91147.95 

******* 

5.8967 

91147.72 i 

10173 

10001 

******* 

10008 

^ .! 7.0 

79155.21 

6.9071 

79140.78 

******* 

6.9097 

79140.57 1 

10130 

10001 

******* 

10004 

S 8.0 

69884.07 

7.9184 

69871.36 

* 'k 'k 'k 'k * 
1 

7.9200 

69871.20 ! 

iOtOi 

10001 ! 

******* 

10002 

J 9.0 

62530.98 

I 8.9274 

62519.63 

******* : 

8.9283 

62519,51 ! 

10081 

10001 I 

******* 

10002 

{ 10.0 

56567.83 

9.9346 

56557.58 

k k kk kkk ■ 
^ ^ 1 

9.9352 

56557.49 ! 

10066 

10000 ! 

******* 

10001 

12,0 

47506.98 

1 11.9457 

47498.39 

******* ! 

11.9457 

47498.33 ! 

10046 

10000 ! 

******* 

ioooo 

■ 14.0 

40966.29 

! 13.9536 

40958.89 

******* { 

13.9534 

40958.86 J 

10034 

10000 1 

******* 

ioooo 

: i6.o 

36034.89 

! 15.9597 

36028.38 

******* : 

15.9594 

36028.36 1 

10026 

10000 1 

******* 

ioooo • 

: i8.o 

32191.44 

! 17.9645 

32185.63 

kk kkk k k * 

T* ^ ^ ^ 1 

17.9640 

32185.62 1 

10021 

10000 1 

******* 

ioooo 

! 20.0 

29116.97 ■ 

! 19,9683 

29111.72 

******* | 

19.9678 

29111.71 ! 

10017 

10000 I 

^ ^ ^ 

ioooo 

S 25.0 

23603.76 

1 24.9753 

23599.51 

******* | 

24.9748 

23599.50 ! 

10011 

10000 I 

******* 

ioooo 

•J 30.0 

19969.07 

I 29.9802 

19965.47 

******* I 

29.9796 

19965.47 ! 

10 0 08 

10000 1 

******* 

ioooo 

J 35.0 

17417. 12 

I 34.9837 

17413.98 

******* | 

34.9832 

17413.98 ! 

10006 

ioooo : 

******* 

ioooo 

i 40.0 

15547.30 

E 39.9864 

15544.50 

******* | 

39.9860 

15544.50 1 

10005 

10000 ! 

******* 

ioooo 

i 45. 0 

14136.52 

1 44.9887 

14133.97 

******* | 

44.9882 

14133.97 

10004 

ioooo ; 

******* 

ioooo 

i 50.0 

13051.12 

1 49.9905 

13048.77 

******* | 

49.9901 

13048,77 1 

10003 

10000 1 

| ******* 

ioooo 

! 60.0 

11546.86 

! 59,9935 

11544.78 

******* ! 

59.9932 

11544.78 S 

10002 

10000 ! 

! ******* 1 

ioooo 

: 7 o.o 

10642.83 

t 69.9959 

10640.96 

******* : 

69.9957 

10640.96 1 

10002 

10000 ! 

} ******* 

• ioooo 

i 80.0 

10155.91 

79.9980 

10154.08 

******* I 

79.9979 

10154.08 ! 

10002 

10000 ! 

******* ; 

ioooo 

i 90.0 

10001.80 

! 90.0000 

10000.00 

******* ! 

90.0000 

10000.00 1 

10002 1 

E 10000 1 

1 ******* 

ioooo 


I 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), UNITE SANDS (CYBER) AND SPHERICAL SLAB ( VARIAN) CORRECTION HETHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION: .0003860 SCALE HEIGHT: 5610.93 TARGET ALTITUDE l 20000 METERS 


INPUT 

DATA * 

GMD GRAD 

REF SOL 

J CYBER S 

SPHERICAL SLAB J 

ALTITUDE COMPUTED BY 

EACH SOLUTION 

METHOD ! 

S E MEAS 
DEG 

R MEAS ! 

METER l 

E COR 
DEG 

R COR 
METER 

l E COR : 
l DEG 1 

E COR 
DEG 

R COR 2 

METER 

NO CORR 2 
METER 1 

GMD H 
METER 

1 CYB H S 

METER 

VAR H ! 

METER 

• 0.0 

i 

603994.40 \ 

- .8155 

603868.60 

1 1 
i ******* : 

-.6991 

603909. OS ! 

• 

28534 2 

19961 

1 1 
{ ******* ! 

l 

21187 l 

• 5 

518812.71 S 

-.1169 

S18722 . 43 

i ******* : 

-.1202 

518728.25 ! 

25577 2 

20003 

i ******* ; 

19974 S 

1.0 

452108.11 1 

.5090 

452038.47 

; ******* 

.5088 

452039.54 \ 

23869 J 

20003 

******* s 

20002 S 

1.5 

398145.30 S 

1.0955 

398089.06 

; ******* : 

1.0983 

398090.25 2 

22808 2 

20003 

\ ******* l 

20023 ! 

2.0 

353657.15 ! 

1.6581 

353610.24 

2 ******* 2 

1.6601 

353611,30 J 

22109 2 

20002 

: ******* i 

20015 

2.5 

316568.79 l 

2.2052 

316528.71 

2 ******* 2 

2.2044 

316529.44 2 

21628 2 

20002 

S ******* ! 

19998 

3.0 

285374.36 \ 

2.7417 

285339.48 

J ******* J 

2.7387 

285339.84 2 

21284 2 

20000 

! ******* J 

19985 

3.5 

258989.79 i 

3.2707 

258958.97 

2 ******* ! 

3.2677 

258959.03 2 

21034 2 

20000 

2 ******* : 

19987 

4.0 

236497.73 ! 

3.7943 

236470.16 

2 ******* 2 

3.7945 

236470 . 00 1 

20848 2 

20000 

2 ******* 2 

20001 

4.5 

217192.73 S 

4.3137 

217167.82 

S ******* \ 

4.3207 

217167.53 2 

20705 2 

20000 

2 ******* J 

20027 

S.O 

200511.07 1 

4.8299 

200488.37 

2 ******* J 

4.8395 

200488.00 2 

20594 2 

20000 

2 ******* ! 

20034 

6.0 

173298.67 i 

5.8555 

173279.42 

2 ******* 2 

5.8612 

173279.00 ! 

20436 2 

20001 

1 1 

| 1 

20018 

7.0 

152201.00 } 

6.8747 

152184.30 

2 ******* 2 

6.87Q3 

152183.95 2 

20332 2 

20001 

1 1 
| T 1 

20010 

8.0 

135471.75 S 

7.8896 

135457.02 

2 ******* 2 

7.8919 

135456.75 1 

20261 2 

20001 

i i 

20006 

9.0 

121940.42 1 

8.9014 

121927.25 

J ******* J 

8.9030 

121927.04 S 

20209 2 

20000 

i ******* j 

20004 

10.0 

110804.33 S 

9,9111 

110792.42 

I ******* 2 

9.9121 

110792.26 2 

20172 2 

20000 

2 ******* ! 

20002 

12.0 

93623.47 % 

11.9258 

93613.47 

2 ******* J 

11.9263 

93613.38 2 

20121 2 

20000 

2 ******* 2 

20001 

14.0 

81043.11 * 

13.9366 

81034.49 

J ******* • 

13.9367 

81034.43 2 

20089 2 

20000 

I ******* 1 

20001 

16.0 

71469.62 \ 

15.9447 

71462.04 

2 ******* } 

15.9447 

71462.00 2 

20069 2 

20000 

2 ******* I 

20000 

18.0 

63960.69 1 

17.9512 

63953.92 

******* ; 

17.9511 

63953.89 i 

20054 S 

20000 

2 ******* 1 

20000 

20 . 0 

57926.86 \ 

19.9564 

57920.73 

; ******* ; 

19.9562 

57920.71 2 

20044 2 

20000 

J ******* { 

20000 

25.0 

47053.05 1 

24.9660 

47048.08 

2 ******* { 

24.9657 

47048.07 2 

20028 2 

20000 

I ******* { 

20000 

30.0 

39851.40 1 

29.9725 

39847.20 

J ******* 2 

29.9723 

39847.19 2 

20019 2 

20000 

) ******* 2 

20000 

35.0 

34781.81 ! 

34.9774 

34778.15 

2 ******* 2 

34.9771 

34778.14 2 

20013 2 

20000 

{ ******* 2 

20000 

40. 0 

31061.24 S 

39.9811 

31057.97 

’2 ******* J 

39.9809 

31057.97 i 

20010 2 

20000 

J ******* 2 

20000 

45.0 

28251.04 i 

44.9842 

28248.07 

2 ******* 2 

44.9839 

28248.06 2 

20008 2 

20000 

2 ******* { 

20000 

50.0 

26087.34 i 

49.9868 

26084.59 

J ******* : 

49.9865 

26084.59 2 

20006 2 

20000 

{ ******* } 

20000 

60.0 

23086.57 i 

59.9909 

23084.14 

2 ******* 2 

59.9907 

23084.14 2 

20004 1 

20000 

1 ******* • 

20000 

70.0 

21282 . 18 i 

69.9943 

21279.94 

2 ******* 1 

69.9942 

21279.94 ! 

20003 2 

20000 

2 ******* ; 

20000 

80.0 

20309.87 \ 

79.9972 

20307.73 

2 ******* 2 

79.9972 

20307.73 2 

20002 J 

20000 

2 ******* 2 

20000 

90.0 

20002.11 : 
i 
i 

90.0000 

20000.01 

2 ******* { 
2 i 

90.0000 

20000.01 2 
1 
1 

20002 2 
2 

20000 

2 ******* ! 

• 1 

1 1 

20000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB (VAR I AN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0003860 SCALE HEIGHT! S610.93 TARGET ALTITUDE! SOOOO METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB J 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

1 

METHOD ! 

E MEAS 
DEG 

R MEAS 
METER 

E COR 
DEG 

R COR 
METER 

E COR 
. DEG 

E COR* 
DEG 

R COR t 

METER ! 

NO CORR 
METER 



GMD H 
METER 



CYB H 
METER 

I 

< 

1 

VAR H J 

METER ! 

0.0 

910793.99 

9365 

910663.45 

******* • 

-.8368 

i 

910708.26 1 

64702 



49932 



******* 

\ 

1 

« 

51510 

.5 

824342.11 

-.2127 

824249.15 

******* 

-.2736 

824257.31 S 

60181 



50003 



******* 

1 

1 

49135 

i .0 

753907.56 

.4295 

753836.12 

******* 

.3913 

753835.35 S 

57455 



50003 



******* 

1 

1 

49504 

1.5 

694042.58 

1.0274 

693984.98 

******* 

1.0051 

69398S . 2i i 

55686 



50 0 03 



******* 

1 

1 

49735 

2.0 

641883.19 

1.5985 

641835.18 

******* 

1 .5831 

641835.69 i 

54468 



50003 



******* 

1 

1 

49831 

2.5 

595807.39 

2.1522 

595766.38 

******* 

2.1386 

59S7£>£> . 76 ! 

53592 



50002 



******* 

1 

1 

49862 

3.0 

554742.22 

2.6941 

554706.52 

******* 

2.6812 

S54706 . 63 \ 

52938 



49998 



******* 

> 

1 

49875 

3.5 

518008.85 

3.2274 

517977.30 

******* 

3.2170 

517977. IS ! 

52443 



49999 



******* 

1 

1 

49905 

4.0 

484997.01 

3.7546 

484968.78 

******* 

3.7498 

484968.45 J 

52059 



49999 



******* 

• 

1 

49959 

4.5 

455242.85 

I* 4.2772 

455217.33 

******* 

4.2817 

455216.90 I 

51754 



49999 



******* 

( 

t 

50035 

5.0 

428356.61 

4.7961 

428333.34 

******* 

4.8047 

428332.85 ! 

51510 



50000 



******* 

• 

1 

50063 

6.0 

381895.22 

5.8261 

381875.4 7 

******* 

5.8311 

381874.96 1 

51147 



50000 



******* 

i 

50033 

7.0 

343427.70 

6,8487 

343410.56 

******* 

6.8519 

343410.14 J 

50896 



50000 



******* 

1 

1 

50019 

8.0 

311282.20 

[• 7.8663 

311267.08 

******* 

7,8684 

311266.75 1 

50716 



50000 



******* 

1 


9.0 

284171.01 

8.8804 

284157.48 

******* 

8.8819 

284157.23 ! 

50584 



50000 



******* 

1 

1 

50007 

10 . 0 

261099.36 

! 9.8920 

261087.12 

******* 

9.8930 

261086.94 i 

50484 



50000* 



******* 

• 

1 

50005 

12.0 

224159.63 

11.9097 

224149.35 

******* 

11.9102 

224149.24 ! 

50346 



50000 



******* 

1 

• 

50002 

14.0 

196100.68 

1 13.9226 

196091.82 

******* 

13.9229 

196091.75 S 

50258 



50000 



******* 

1 

1 

50001 

16.0 

174203.28 

IS. 9325 

174195.48 

******* 

15.9326 

174195.43 » 

50198 



50000 



£ ^ 

1 

1 

50001 

18.0 

156717.90 

1 17.9403 

156710.93 

******* 

17.9403 

156710.90 J 

50157 



50000 



******* 

1 

1 

50000 

20.0 

142482.17 

19.9466 

142475.87 

******* 

19.9466 

142475.84 l 

50126 



50000 



******* 

1 

1 

50000 

25.0 

116446.31 

! 24.9583 

116441.20 

******* 

24.9582 

116441.19 1 

50079 



50000 



******* 

1 

1 

50000 

30.0 

98962.21 

29.9663 

98957.89 

******* 

29.9661 

98957.88 i 

50052 



50000 



******* 

1 

1 

5000 0 

35.0 

86554.25 

1 34.9722 

86550.48 

******* 

34.9721 

86550.48 \ 

50036 



50000 



******* 

1 

1 

50000 

40.0 

77401.43 

39.9768 

77398.07 

******* 

39.9767 

77398.06 ! 

50026 



50000 

1 


******* 

1 

1 

50000 

45.0 

70464.56 

! 44.9805 

70461.50 

| ******* 

44.9804 

70461.50 ! 

50019 



50000 



******* 

1 

1 

50000 

50.0 

65110.90 

49.9837 

65108.08 

******* 

49.9836 

65108.07 1 

50014 



SOOOO 



******* 

1 

1 

SOOOO 

60,0 

57669.38 

! 59.9888 

57666.88 

I ******* 

59.9887 

57666.88 ! 

50008 



50000 



******* 

i 

500 00 

70.0 

53186.20 

69.9929 

53183.90 

******* 

69.9929 

53183.90 J 

50004 



50000 



******* 

1 

1 

50000 

80.0 

S0767.93 

I 79.9966 

50765.73 

! ******* 

79.9965 

50765.73 ! 

50003 



50000 



******* 

t 

1 

50000 

90.0 

50002.17 

1 

90.0000 

50000.01 

1 

******* 

t j 

90,0000 

[ 

50000.00 : 

i 

50002 



50000 


! 

******* 

• 

I 

SOOOO 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 


COMPARISON OF GRADIENT (GMD) , UNITE SANDS (CYBER > AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON MEASURED VALUES CONTAINED IN JSC INTERNAL NOTE 7S-FM-60 'REFRACTION CORRECTIONS FOR AN EXPONENTIAL ATMOSPHERE' 
SEA LEVEL MODULUS OF REFRACTION! .0003860 SCALE HEIGHT! 5610.93 TARGET ALTITUDE i 100000 METERS 


INPUT 

DATA i 

GMD GRAD 

REF SOL 

CYBER i 

SPHERICAL SLAB ! 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 

R MEAS 1 

E tOR 

R COR I 

E COR 1 

E COR 

R COR 1 

NO CORR 


GMD H 


CYB H » 

VAR H 

DEG 

METER ! 

DEG 

METER i 

DEG S 

DEG 

METER • ! 

METER 


METER 


METER ! 

METER 

0.0 

» 

1250390.50 1 

-1.0016 

: 

1250257.80 i 

i 

« 

* ****** \ 

-.9121 

2 

1250304.58 S 

121409 


99901 


******* : 

101832 

.5 

1163337.45 ! 

-.2605 

1163243.51 ! 

******* J 

-.3544 

1163252.57 ! 

115204 


100003 


******* *, 

98127 

1 .0 

1091120.67 ! 

.3923 

1091048.71 ! 

******* | 

.3330 

1091046.76 ! 

111400 


100004 


******* : 

98891 

1.5 

1028349.90 l 

.9972 

1028292.01 

******* } 

.9614 

1028291.64 ! 

108890 

\ 

100003 


******* : 

99370 

2.0 

972251.85 l 

1 .5733 

972203.66 ! 

******* ; 

1.5487 

972203.85 ! 

107134 

\ 

100003 


******* s 

99592 

2.5 

921320.18 i 

2.1306 

921279.05 

******* : 

2.1104 

921279.25 ! 

105851 

* 

* 

100002 


******* : 

99684 

3.0 

874609 . 93 ! 

2.6752 

874574.15 S 

******* ; 

2.6575 

874574.15 S 

104877 

i 

99996 


******* ; 

99731 

3.5 

831588.85 

3.2107 

831557.24 

******* : 

3.1966 

831557.02 ! 

104129 

• 

99997 


******* ; 

99797 

4.0 

791780.91 ! 

3.7396 

791752.63 

******* j 

3.7323 

791752.25 S 

103537 

I 

99998 


******* ! 

99899 1 

4.5 

754853.54 

4.2635 

754827.99 

******* •, 

4.2668 

754827.52 1 

103061 

! 

99998 


******* s 

100040 

5.0 

720536.96 ! 

4.7836 

720513.67 

******* ! 

4.7915 

720S13.15 ! 

102672 

• 

i 

99999 


******* 2 

100096 

6.0 

658852.88 

5.8154 

658833.12 

******* ; 

5.8200 

658832.59 I 

102081 

i 

99999 


******* J 

100051 

7.0 

605213.48 1 

6.8394 

605196.33 

******* i 

6.8423 

605195.89 1 

101661 

! 

99999 


******* \ 

100029 

8.0 

558394.60 ! 

7.8581 

558379.47 ! 

******* : 

7.8600 

558379.13 S 

101351 

i 

i 

100000 


******* J 

100017 

9 . 0 

517376.01 S 

8.8730 

517362 . 47 ! 

******* i 

8.8743 

517362.22 I 

101118 

i 

100000 


1 

100011 

10. 0 

481299.06 1 

1 9.8853 

481286.82 

******* i 

9.8862 

481286.63 J 

100938 

2 

100000 


******* 

10 0 0 07 

12.0 

421189.36 ! 

11.9040 

421179.08 ! 

.*■ ^ •+ 1 'k 'to 'k 'it 1 

^ •T' T ^ ^ 1 

11.9045 

421178.96 J 

100682 

i 

• 

100000 


ft 

100003 

14.0 

373544.54 ! 

I 13.9177 

373535.67 

■ 

T* 'r ^ 4* | 

13.9180 

373535.60 i 

100515 

i 

100000 


******** ^ 
^ ^ 4^ T ^ 1 

100002 

16.0 

335165.07 1 

15.9282 

335157.27 ! 

******* l 

15.9284 

335157.22 i 

100400 

i 

i 

100000 


******* ; 

i 0 0 0 0 1 

18.0 

303778.21 1 

1 17.9365 

303771.24 

******* | 

1 17.9366 

303771.21 J 

100318 

i 

100000 


******* J 

i 0 0 0 0 0 

20.0 

277753.64 ! 

19.9432 

277747.34 ! 

******* J 

19.9432 

277747.31 J 

100257 

« 

100000 


******* ; 

10000 0 

25.0 

229124.13 ! 

24.9556 

229119.02 

******* j 

! 24.9555 

229119.01 ! 

100161 

i 

100000 


******* J 

10000 0 

30.0 

195776.10 J 

29.9641 

195771.78 : 

******* : 

29.9640 

195771.7 7 i 

100107 

t 

100000 


******* : 

100000 

35.0 

17180B.22 ! 

34.9703 

171804.45 

i ******* i 

! 34.9703 

171804.45 1 

100074 

{ 

100000 


******* 2 

10 0000 

40 . 0 

153983.17 1 

39.9752 

153979.81 

! ******* { 

1 39.9752 

153979.80 ! 

100052 

i 

100000 


******* { 

100000 

45. 0 

140398.84 

44.9792 

140395.78 

i it 'k it ^ 'A' 1 

1 ^ ^ ^ T 4^ l 

! 44.9792 

140395.78 J 

100038 

i 

i 

100000 


******* 2 

100000 

50.0 

129874.11 1 

49.9826 

129871.29 

J ******* j 

1 49.9825 

129871.28 ! 

100027 

i 

100000 


******* ; 

100000 

60 . 0 

115190.67 

59.9880 

115188.17 

: ******* 

! 59.9880 

115188.17 ! 

100014 

J 

100000 


******* 2 

100000 

70.0 

106316.53 ! 

69.9924 

106314.23 

I ******* | 

! 69.9924 

106314.23 ! 

100007 

i 

100000 


******* I 

i 0 0 0 0 0 

80 . 0 

101521.64 

79.9963 

101S19.44 

j ******* 

! 79.9963 

101519.44 i 

100003 

i 

i 

100000 


******* i 

100000 

90.0 

i . _ _ 

100002.17 : 

90.0000 

100000.01 

! ******* 
1 

! 90.0000 
1 

100000.00 ! 
1 
1 

100002 

• 

i 

100000 


******* 2 
i 
i 

100000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



APPENDIX D 


TABULATED REFRACTION COMPARISON DATA 
FOR EDWARDS EHA-75 HOT DAY CONDITIONS 



TABLE D-I 



LEATHER DATA 

FOR EDWARDS 

EHA-75 HOT DAY 

ALTITUDE 

(FT) 

TEMPERATURE 
(DEG C) 

DEW POINT 
(DEG C> 

PRESSURE N X 10E6 

<MB) 


2316 

43.69995 

8.84998 

929.00000 

270.30353 

3000 

38.58756 

7.11333 

903.25562 

264.14508 

4000 

36,85410 

6.14978 

891 . 03394 

260.40588 

5000 

33.24156 

3.34049 

857.79834 

249.10526 

6000 

29.12725 

.12280 

819.52612 

236.17365 

7000 

26.56799 

-1.93021 

794.60022 

227.98520 

8000 

24.10647 

-4.03682 

768.15747 

219.90646 

9000 

21.49864 

-6.32584 

739.38721 

211.39047 

10000 

19,05116 

-8.47149 

713.05518 

203.66263 

1 1 0 0 0 

16.59399 

-10.61668 

688. 07678 

196.55545 

12000 

14,05948 

-12.81905 

663.23145 

189.72156 

13000 

11.61666 

-14.90845 

639.75146 

183.34998 

14000 

9.20186 

-16,92825 

617.23413 

177.32986 

15000 

6.64556 

-19.08471 

594.35840 

171.38351 

16000 

4.16272 

-21.27731 

572.38318 

165.70724 

17000 

1,87212 

-23.51741 

551 .87170 

160.34540 

18000 

-.46410 

-26.06773 

531 . 02637 

154.89380 

19000 

-2.75558 

-28.60963 

510 .69012 

149.60373 

20000 

-4.89093 

-30.78820 

491.85645 

144.75958 

21000 

-7.06930 

-32.70390 

473.03528 

140 . 02933 

22000 

-9.26616 

-34.49268 

454.41113 

135.40790 

23000 

-11.34800 

-36.20806 

437.01611 

131 . 08182 

24000 

-13.44421 

-38.00616 

420.13660 

126.84921 

25000 

-15.62985 

-39.89327 

403.19788 

122.58925 

26000 

-17.74000 

-41.72385 

387.11890 

118.53600 

27000 

-19.75973 

-43.48828 

371.99591 

114.70740 

28000 

-21.89243 

-45,36777 

356.70337 

110.80807 

29000 

-24.00026 

-47.22755 

341.96582 

107.03584 

30000 

-25.98938 

-48.97205 

328.30975 

103.53395 

31000 

-28.10549 

-50,81854 

314.59778 

99.99405 

32000 

-30,26072 

-52.69841 

301 . 19305 

96.51865 

33000 

-32,30277 

-54.48528 

288.76862 

93.29465 

34000 

-34.44572 

-56.34369 

276.54724 

90 .10873 

35000 

-36.68972 

-58.24686 

264.43378 

86.93344 

36000 

-38,83938 

-60 .00636 

253.05801 

83.94058 

37000 

-40.94783 

-61.59187 

242.22083 

81 . 06636 

38000 

-43.17965 

-63.13728 

231.34473 

78.14810 

39000 

-45,36091 

-64.62108 

220.97690 

75.34836 

40000 

-47.42838 

-66.07417 

211.32605 

72.72676 

41000 

-49.62975 

-67.62959 

201.63461 

70,05484 

42000 

-51.84161 

-69.18915 

192.26611 

67,45100 

43000 

-53.93611 

-70.65814 

183.60962 

65.04021 

44000 

-56.14576 

-72.20313 

175.02695 

62,61802 

45000 

-58.38364 

-73.77621 

166.62195 

60.21470 

46000 

-60.44167 

-75.24471 

158.81183 

57.96035 

47000 

-62.46535 

-76.62076 

151 .26779 

55.74521 

48000 

-64.63737 

-77.89751 

143.78864 

53.52798 

49000 

-66.91588 

-78.96187 

136.73184 

51.45824 

50000 

-69.38409 

-79.65543 

130 . 10373 

49.55390 
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TABLE D-I 


Continued 



WEATHER DATA 

FOR EDUARDS 

EHA-75 HOT DAY 


ALTITUDE 

(FT) 

TEMPERATURE 
(DEG C) 

DEU POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 1 0E6 

51000 

-71 . 93837 

-79.95189 

123.48726 

47.61012 

52000 

-73.92885 

-80.07860 

117.17973 

45.63460 

53000 

-74.58440 

-80.21500 

111.38791 

43.56802 

54000 

-73.34724 

-80.22905 

10S. 72054 

41.12403 

55000 

-71.31264 

-80.17506 

100.29747 

38.60095 

56000 

-69.71280 

-80.15002 

95.32890 

36.36906 

57000 

-68.36600 

-80.15002 

90.59485 

34.33443 

58000 

-66.95903 

-80.15002 

86.05518 

32,40004 

59000 

“65.57408 

-80.15002 

81.84933 

30.60943 

60000 

“64.15100 

-80.15002 

77.90450 

28.93344 

61000 

-62.67793 

-80.15002 

74.08270 

27.32929 

62000 

-61.36696 

-80.15002 

7 0.49979 

25.84626 

63000 

-60.41143 

-80.15002 

67.18153 

24.50804 

64000 

-59,77395 

-80.15002 

63.94312 

23,25805 

65000 

-59.26440 

-80.15002 

60.86837 

22. 0931S 

66000 

-58,69947 

-80.15002 

58.03733 

21.00944 

67000 

-58.05930 

-80.15002 

55.28382 

19.95426 

68000 

-57.40619 

-80.15002 

52.63624 

18.94324 

69000 

-56.78934 

-80.15002 

50.19029 

18,01028 

70000 

-56.16160 

-80.15002 

47.85101 

17.12111 

71000 

-55.50677 

-80.15002 

45.57531 

16.25982 

72000 

-54.87612 

-80.15002 

43.44771 

15.45611 

73000 

.-54,26505 

-80.15002 

41.45888 

14.70639 

74000 

-53.61657 

-80.15002 

39.50738 

13.97437 

750 0 0 

-52.97717 

-80.15002 

37.66196 

13.28384 

76000 

-52.37175 

-80.15002 

35.96156 

12.64812 

77000 

-51.72646 

-80.15002 

34.29820 

12.02902 

78000 

-51 . 07536 

-80.15002 

32.70680 

11.43848 

79000 

-50.46436 

-80.15002 

31,23968 

10.89460 

80000 

-49.82896 

-80.15002 

29.81631 

10.36910 

81000 

-49.17453 

-80.15002 

28.43674 

9.86171 

82000 

-48.55239 

-80,15002 . 

27.15546 

9.39114 

83000 

-47.93436 

-80.15002 

25.94243 

8,94686 

84000 

-47.28407 

-80,15002 

24.75444 

8.51371 

85000 

-46.65055 

-80 . 15002 

23.63672 

8.10699 

. 86000 

-46.04258 

-80 . 15002 

22.59821 

7.72960 

87000 

-45.39286 

-80 .15002 

21.57514 

7.35950 

88000 

-44.74556 

-80.15002 

20.60023 

7.00774 

89000 

-44. 13775 

-80.15002 

19.70158 

6.68383 

90000 

-43.49626 

-80 . 15002 

18.82100 

6.36783 

91000 

-42.84356 

-80 . 15002 

17.97160 

6.06404 

92000 

-42.22903 

-80 . 15002 

17. 18723 

5.78379 

93000 

-41.60152 

-80 . 15002 

16.43345 

5.51540 

94000 

-40.95004 

-80 . 15002 

15.69908 

5.25495 

95000 

-40.32507 

-80 . 15002 

15.01326 

5. 01210 

96000 

-39.71589 

-80.15002 

14.37071 

4.78499 

97000 

-39. 06644 

-80.15002 

13.73852 

4.56246 

98000 

-38.42149 

-80.15002 

13.13915 

4.35189 

99000 

-37.80067 

-80 . 15002 

12.58263 

4.15653 

100000 

-37. 13890 

-80 . 15002 

12.03220 

3.96417 
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COMPARISON OF GRADIENT (GMD). UNITE SANDS (CYDER) AND SPHERICAL SLAD ( VAR IAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR EDUARDS EHA-75 HOT DAY SEA LEVEL INDEX OF REFRACTION i .0002939 SCALE HEIGHT i 72G4.10 METERS 



; INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB J 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 


J E MEAS 

R NEAS 

E COR 

R COR 

E COR 

E COR 

R COR 

NO CORR 

GMD H 

CYB H 

VAR H 


i 'DEG 

•METER 

DEG* 

METER 

DEG 

DEG 

METER 

METER 

METER 

METER 

METER 


i 

i o.o 

* 

130488.61 

* 

-.1467 

* 

130451.73 

— . 1577 

-.1490 

130444.37 

1335 

* 

1000 

976 

994 


; .5 

75757.79 

.4164 

75736.73 

* .4126 

.4134 

75735.50 

1111 

1000 

995 

996 


: l.o 

49121.62 

.9463 

49108.03 

.9441 

.9466 

49107.77 

1046 

1000 

998 

1000 


; i.s ■' 

35374.34 

r 1.4616 

35364.57 

1.4599 

1.4623 

35364.55 

1024 

1000 

999 

■ 1000 


i 2.0 

27384.03 

1.9705 

27376.47 

1.9690 

1.9707 

27376.51 

1014 

1000 

1000 

1000 


• ! 2 . S 

• 222SS.74 

2.4761 

22249.60 

2.4749 

2.4759 

• 22249.65 

1010 

1000 

1000 

1000 


! 3.0 

18713.66 

2.9801 

18708.49 

2.9789 

2.9795 

18708.54 ! 

1007 

1000 

1000 

1000 


r 3.5 

16130.52 

-3.4829 

“16126.07 

3,4818 

3.4822 

16126.11 

1005 

1000 

1000 

1000 


: 4.o 

14167.60 

3. 9851 

14163.70 

3.9841 

3.9846 

14163.72 

1004 

1000 

1000 

1000 


: - 4.5 

12627.46 

4.4868 

12623.98 

4.4858 

4.4868 

12624.00 

1003 

1000 

1000 

1000 


! 5.0 

11387.79 

4.9882 

11384.65 

4.9872 

4.9877 

11384.67 

1003 

1000 

1000 

1000 


: 6.o 

9517.44 

r S , 9903 - 

9514.82 

5.9893 

5.9897 

9514.83 

1002 

1000 

1000 

1000 

o 

! 7.0 

8175.00 

6.9918 

8172.75 

6.9908 

6.9911 

8172.76 

1001 

1000 

1000 

1000 


r 8,o 

-7165.11- ! 

\ 7.9929 

• 7163.14 

[ 7.9920 

7.9922 

7163.15 

! 1001 

1000 

1000 

1000 

M-fc 

5 9.0 

6378.61 I 

8.9937 

6376.85 

! 8.9929 

8.9931 

6376.85 

1001 

1000 

1000 

1000 


! 10.0 

5748.-92 ! 

*9.9944 

* 5747.34' 

! 9.9936 

[ 9.9938 

5747.34 I 

i" 1001 

• 1000 

1000 

1000 


1 12.0 

4804.48 i 

11.9955 

4803.16 

! 11.9947 

11.9948 

4803.16 ! 

1001 

1000 

1000 

1000 


\ 14 ;o 

4130.56 ! 

’13.9963 

4129.42 

1 13.9955 

\ 13.9956 

4129.42 ! 

l * 10 01 ’ 

1000 

1000 

1000 


! 16.0 

3626.11 | 

15.9968 

3625.12 

! 15.9961 

15.9962 

3625.12 I 

1000 

1000 

1000 

1000 


: i8.o * 

3234.88 ! 

17.9973 

3233.99- 

1 17.9965 

! 17.9966 

*3233.99 ! 

! 1000 

1000 

1000 

1000 


j 20.0 

2923.14 J 

19.9976 

2922.34 

! 19.9969 

19.9970 

2922.34 1 

1000 

1000 

1000 

1000 


1 25.0 

2366.05 ! 

24 *. 9983 

2365.40 

! 24.9976 

1 24.9976 

2365.40 1 

l * 1000 

1000 

1000 

1000 


1 30.0 

1999.99 1 

29.9987 

1999.43 

! 29.9980 

29.9981 

1999.44 ! 

: iooo 

1000 

1000 

1000 


J* 35.0 

‘ 1743.53 ' 

34.9990 

* 1743.10- 

J 34.9984 

34.9984 

1743.10 

! • 1000 

10 0 0 

1000 

10 0 0 


! 40.0 

1555.95 ! 

39.9992 

1555.52 

! 39.9987 

39.9987 

1555.52 i 

I 1000 

1000 

100 0 

10 0 0 


i 45.0 

1414.49 

44.9994 

1414.10 

J 44,9989 

44.9989 

1414.10 

: iooo 

1000 

1000 

10 0 0 


! 50.0 

1305.54 ! 

49.9995 

1305.18 

1 49.9991 

49.9991 

1305.18 ! 

: iooo 

1000 

10 0 0 

10 0 0 


S 60.0 

1154.86 : 

59.9997 

1154.55 

! 59.9993 

59.9994 

1154.55 

1 1000 

1000 

100 0 

10 0 0 


i 70.0 

1064.36 j 

69.9998 

1064.07 

! 69,9996 

69.9996 

1064.07 i 

i 1000 

1000 

1000 

10 0 0 


: 80 ,o 

1015.61 

• 79.9999 * 

1015.34 

! 79.9998 

79.9998 

1015.34 

: iooo 

1000 

10 00 

10 0 0 


i 90.0 

! 

1000.19 ! 

90.0000 

999.92 

1 90.0000 

i 

i 

90 .0000 

999.92 ! 

! 1000 

i 

i 

1000 

1000 

1000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIhITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD) , WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
, • BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC. INTERNAL NOTE 7S-FM-6Q 


WEATHER DATA FOR EDWARDS EHA-75 HOT DAY SEA LEVEL INDEX OF REFRACTION i ,0002939 SCALE HEIGHT i 7264.10 METERS 


\ INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB 

1 . ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 

R MEAS 

E COR 

R COR 

E COR 

E COR 

R COR 

i NO CORR 


GMD H 


CYB H ! 

VAR H 

! DEG 

METER 

DEG ' 

* METER* 

DEG 

DEG 

METER 

! METER 


METER 

i 

METER S 

METER 

! 0.0 

184047.44 

i 

-.2035 

183996.95 

-.2224 

-.2051 

183993.28 

1 

I 2655 

• 

2000 

1 

J 

1941 I 

1995 

r .s 

124157.76 * 

l .3658 * 

124124.54 

* .3608 

.3604 

124123.04 

i 2291 


2000 


1990 ! 

1988 

! 1 .0 

88093.29 

.9058 

88069.95 

.9041 

.9077 

88069.78 

t 2145 


2000 


1998 1 

2003 

i . 5 

*66367.38* 

i . 4295 

** 66349.87 * 

1.4287 

1.4323 

66350 . 10 

2082 


2000 


2000 : 

2003 

: 2 .o 

52583.71 

1.9444 

52569.87 

1.9439 

1 .9463 

52570.18 

2052 


2000 


2000 i 

2002 

• 2.S ' 

43292.05 ' 

2.4544 

43280.67 

2,4540 

* 2.4552* 

43280.97 

2035 


2000 


2000 ! 

2001 

: 3.o 

36685.57 

2.9615 

36675.93 

2.9611 

2.9616 

36676,19 

2025 


2000 


2000 1 

2000 

! 3.5 

31779.22 

■3.4668 

31770.87 * 

3.4663 

3.4666 

* 31771.09 

2019 


2000 


2000 ! 

2000 

: 4.0 

28005.88 

3.9708 

27998.53 

3.9704 

3.9710 

27998.71 

2015 


2000 


2000 1 

2000 

! 4/5 

25020,29 

4.4740 

25013.72 

* 4.4736 

4.4750 

-25013,88 

2012 


2000 


2000 { 

2000 

S 5.0 

22602.95 

4.9766 

22597.02 

4.9762 

4.9769 

22597.15 

2010 


2000 


2000 ! 

2000 

i 6.0* 

18933.71 

* S . 9806 

18928.74 

5.9801 

5.9805 

18928.84 

2007 ‘ 


2000 


2000 ! 

2000 

! 7.0 

16285.56 

6.9834 

16281.29 

6.9829 

6.9832 

16281.37 

2005 


2000 


2000 \ 

2000 

: 8.o 

14287,26 

7.9856* 

14283.51 ! 

[ 7.9850 

7.9852 

* 14283.58 

2004 


2000 


2000 i 

2000 

; 9.0 

12726.84 

! 8.9873 

12723.50 1 

8.9867 

8.9868 

12723,56 

2003 


2000 


2000 { 

2000 

! 10.0 

11475.71 

\ 9.9886 

' 11472.70 I 

t 9.9881 

9.9881 

11472.76 

2003 


2000 


2000 : 

2000 

; i2.o 

9596.12 

! 11.9907 

9593.60 I 

11.9901 

11.9901 

9593.65 

2002 


2000 


2000 I 

2000 

S 14.0 

8253*. 05 

I 13.9921 

8250.88 ! 

! 13,9915 

13,9916 

8250,92 

2002 


2000 


2000 ! 

2000 

! 16.0 

7246.88 

15.9932 

7244.98 1 

15.9926 

15.9927 

7245.02 

2001 


2000 


2000 ! 

2000 

! 18.0 

6466.19 

1 17.9941 

6464.50 1 

t 17.9935 

17.9935 

6464.53 

2001 


2000 


2000 J 

2000 

1 .20.0 

5843.58 

! 19.9948 

5842.05 

19,9942 

19.9942 

5842.08 

2001 


2000 


2000 1 

2000 

J '25.0 

4730.78 

! 24.9961 

4729.54 

! 24.9955 

24.9955 

4729.56 

2001 


2000 


2000 1 

2000 

i 30.0 

3999.40 

29.9969 

3998.35 

29.9963 

29.9963 

3998.37 

2001 


2000 


200 0 i 

2000 

! 35.0 

3486.88 

1* 34.9975 * 

3485.97 1 

! 34.9970 

34.9970 

3485.99 

2001 


2000 


2000 J 

2000 

1 40.0 

3111.60 

39.9980 

3110.79 1 

39.9975 

39,9975 

3110.80 

2001 


2000 


2000 \ 

2000 

I 45.0 

2828.78 ‘ 

I 44.9983 ‘ 

2828.04 I 

44.9979 

44.9979 

2828.05 

2001 


2000 


2000 ! 

20 0 0 

• 50.0 

2611 . 12 

49,9986 

2610.44 1 

49.9982 

49.9982 

2610.45 

2000 


2000 


200 0 : 

2000 

! 60.0 

2309.84 

1 59,9991 

*2309.24 J 

59.9988 

59.9988 

2309.25 

2000 


2000 


2000 ! 

20 0 0 

! 70.0 

2128.86 

69.9995 

2128.30 J 

69.9992 

69.9992 

2128.31 

2001 


2000 


2001 J 

20 0 0 

1 80.0 

2031 .21 

! 79,9997 * 

2030.68 J 

79,9996 

79,9996 

* 2030.69 

2000 


2000 

l 

2000 J 

2000 

i 90.0 

t 

* 

« 

2000.37 

90.0000 

1999.85 J 

** | 

90,0000 

90.0000 

1999.86 

2000 


2000 


2000 i 
• 
1 

2000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 










COMPARISON OF GRADIENT (GMD) , WHITE SANDS (CYDER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAhE MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 
WEATHER DATA FOR EDWARDS EHA-7S HOT DAY SEA LEVEL INDEX OF REFRACTIONi .000293? SCALE HEIGHT! 7284.10 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

1 CYBER i 

SPHERICAL SLAB ! 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E MEAS 

R MEAS 

E COR 

R COR 

i E COR 

E COR 

R COR 1 

NO CORR 

GMD H 

CYB H 

MAR H 

DEG 

METER 

- DEG 

METER 

t DEG 

DEG 

METER i 

METER 

METER 

METER 

METER 

0.0 

288718.61 

-.3036 

288646.65 

i 

i -.3489 

-.2533 

\ 

288652.11 i 

6531 

5000 

4775 

5253 

.5 

224057.01 

.2733 

224003.69 

i ' .2625 

.2760 

224003.71 i 

5888 

5000 

4960 

5011 

i.O 

176990.67 

,8249 

176949.66 

I .8249 

.8317 

176950.51 S 

5542 

5000 

5002 

5021 

' 1.5 

'143123.47 ’ 

1.3603 

143090.81 

1 1.3637 

1.3682 

143092,06 i 

5350 

5000 

5010 

5020 

2.0 

118580.41 

1.8852 

118553.59 

} 1.8896 

1.8916 

118554.88 S 

5239 

5000 

5010 

5013 

2.5 

100449.11 

*2.4034 

100426.52 

l 2.4077 

2,4075 

100427.70 l 

5170 

5000 

5009 

5007 

3.0 

86730.01 

2.9169 

86710.5 7 

: 2.9210 

2.9194 

86711.60 S 

S127 

5000 

5007 

5004 

3.5 

76095.39 

3.4274 

76078.38 

i 3.4312 

3.4292 

* 76079.25 i 

5097 

5000 

5006 

5002 

4.0 

67664.99 

3.9356 

67649.89 

! 3.9391 

3.9380 

67650.63 I 

5077 

5000 

5005 

5003 

- 4.5 * 

60847.22 

4,4423 

60833.66 

1 4.4454 

4.4463 

60834.28 \ 

5062 

5000 

5004 

50 04 

5.0 

55236,20 

4.9477 

55223.89 

! 4.9506 

4 , 9516 

55224.43 ! 

50S1 

5000 

5004 

5004 

6.0 * 

• 46575.89 

5,9561 

46565.53* 

! 5.95BS 

5.9586 

46565.94 ! 

• 5037 

5000 

5003 

5002 

7.0 

40227.62 

6.9623 

40218.68 

i 6.9643 

6.9639 

40219.02 I 

5027 

5000 

5003 

5001 

* 8.0" ■ 

35388.37 

■ 7.9670- 

35380.51 

! 7.9687 

7.9681 

35380.80 I 

'5021 

5000 

5002 

5001 

9.0 

31584.05 

8.9707 

31577.03 

1 8.9722 

8.9715 

31577.29 i 

5017 

5000 

5002 

5000 

10 . 0 

28518.66' 

9.9737 

28512.33 

I 9.9750 

1 9.9742 

28512.57 ! 

5014 

5000 

5002 

5000 

12.0 

23891 . 02 

11.9782 

23885.72 

! 11.9792 

11.9785 

23885.93 i 

5010 

5000 

5002 

5000 

■* 14.0 

20570 . 08 

13.9815 

' 20565.52 

! 13.9822 

1 13.9816 

20565.70 ! 

5008 ' 

5000 

5001 * 

' 5000 

16. 0 

18075.60 

15.9839 

18071.59 

i 15.9845 

15.9839 

18071.75 J 

5006 

5000 

5001 

5000 

* 18*. 0 

16136.38 

17.9859 

16132.80 

1 17.9863 

! 17.9858 

16132.95 1 

5005 

5000 

5001 

5000 

20 . 0 

14587.95 

19.9874 

14584.72 

S 19.9878 

19.9873 

14584.85 1 

5004 

5000 

5001 

5000 

25.0 

11816.62 

24.9903 * 

11814.00 

r 24.9905 

t 24.9900 

11814.10 ! 

SO 03 

5000 

5001 

500 0 

30 . 0 

9992.75 

29.9922 

9990.53 

! 29.9923 

29.9920 

9990.62 \ 

5002 

5000 

5001 

500 0 

* 35 . 0 

8713.49 

34.9937 

8711.56 

1 34.9937 

I 34,9934 

8711.64 \ 

5002 

5000 

5001 

5000 

40 , 0 

7776.81 

39.9947 

7775.09 

l 39.9947 1 

! 39.9945 

7775.16 1 

5002 

5000 

500 1 

50 0 0 

~ 45 . 0 

7070.36 

44.9956 

7068.79 

J 44,9956 

! 44.9953 

7068,85 ! 

5001 

5000 

5001 

50 00 

50 . 0 

6526 . 96 

49.9964 

6525.51 

! 49.9963 

1 49.9961 

6525.57 i 

5001 

5000 

50 01 

50 0 0 

60 . 0 

5774.11 

59 . 9975 

5772.84 

1 59.9974 

! 59.9973 

5772.89 J 

5001 

5000 

5001 

50 00 

70 . 0 

5321 . 7S 

69.9985 

5320.57 

1 69.9984 

69.9983 

5320.62 { 

5001 

5000 

50 01 

c n n n 

80 , 0 

5078 . 16 

79.9993 

5077.03 

l 79.9992 

! 79.9992 

5077.08 S 

5001 

5000 

50 01 

jUUU 

50 0 0 

90 . 0 

5001.07 1 

1 

90.0000 

1 

4999.96 

1 90.0000 

90.0000 

i 

5000.01 1 
! 

5001 

I 5000 

■ 5001 

i 

5000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER > AND SPHERICAL SLAB ( VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION' SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR EDUARDS EHA-7S HOT DAY SEA LEVEL INDEX OF REFRACTION! .000293? SCALE HEIGHT! 7284.10 METERS 


INPUT 

DATA 

GHD GRAD 

REF SOL S 

CYDER 

SPHERICAL SLAB I 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E MEAS 

R hEAS 

E COR 

R COR S 

E COR 

E COR 

R COR S 

NO CORR 

GMD H 

CYB H 

VAR H 

DEG 

HETER 

* DEG • 

METER • J 

DEG 

: DEG 

METER S 

METER ‘ 

METER 

METER 

METER 

0.0 

403882.16 

-.3936 

1 

403794.21 \ 

-.4881 

i 

-.2812 

1 

403809.47 J 

1277S 

10000 

934 0 

10792 

.5 ' 

336887.45 

.1896 

336819.00 1 

,1638 

! .2421 

336823.74 i 

11826 

10000 

9852 

10309 

1.0 

283629.66 

.7484 

283574.61 l 

.7471 

.7840 

283578.61 J 

11246 

10000 

9997 

10176 

' 1.5 

‘241435.35 

' 1.2913 * 

241389.91 J 

1.2993 

i i . 3175 

241393.60 ! 

10880 

10000 

10036 

10110 

2.0 

208003.66 

1.8234 

207965.32 1 

1 .8348 

1 .8426 

207968.68 ! 

10642 

10000 

10044 

10 070 

2.5 

181382.71 

2.3480 

181349.76 i 

* 2.3603 

i 2.3616 

181352.74 i 

10482 

10000 

10041 

10043 

3.0 

159998.09 

2.8672 

159969.31 i 

2.8794 

2.8768 

159971.90 i 

10372 

10000 

10036 

10027 

3.5 

142629.58 

3.3824 

142604.11 1 

3.3941 

3.3900 

142606.34 i 

10294 

10000 

10031 

10019 

4.0 

128353.98 

3.8948 

128331.18 \ 

3.9058 

3.9025 

128333.10 t 

10237 

10000 

10027 

10017 

4.5 

116479.61 

4.4049 * 

116459.00 I 

4.4152 

4.4147 

116460.67 S 

10194 

10000 

10023 

10020 

5.0 

106488,82 

4.9134 

106470.04 \ 

4.9230 

4.9236 

106471.49 I 

10162 

10000 

1 0020 

10019 

6.0 

90695.34 

5.9267 

90679,41 S 

5.9351 

5.9338 

90680.56 S 

10117 

10000 

10015 

iOOli 

7.0 

78845.85 

6.9366 

78832.04 1 

6.9440 

6,9419 

78833,01 I 

10088 

10000 

10012 

10007 

8.0 

69670.33 

7.9443 

69658.15 ! 

7.9508 

7.9484 

69659.00 1 

10069 

10000 

10010 

10005 

9.0 

62377.59 1 

8.9503 

62366.70 1 

8,9562 

8.9537 

62367.48 i 

10055 

10000 

10008 

10004 

10.0 

56454.33 ! 

9.9553 

S6444.48 -J 

9.9606 

9.9580 

56445.20 i 

10045 

10000 

10007 

10003 

12.0 

47439.94 i 

11,9628 

47431.68 ! 

11.9672 

11.9648 

47432.29 i 

10032 

10000 

10005 

10002 

14.0 

40923.65 1 

13.9682 

40916.53 i 

13.9719 

13.9698 

40917.07 i 

10024 

10000 

10004 

10 00 1 

16.0 

36006.14 I 

15.9724 

35999.88 J 

15,9756 

15.9736 

36000.36 i 

10018 

10000 

10004 

10001 

18.0 

32171,23 ! 

17.9756 

32165.64 I 

17.9784 

17.9766 

32166.07 I 

10015 

10000 

10003 

10 001 

20.0 

29102.22 ! 

19.9783 

29097.16 ! 

19.9807 

19.9791 

29097.56 \ 

10012 

10000 

10003 

10001 

25.0 

23596.29 ! 

24,9831 “ 

* 23592.19 1 

24.9849 

24.9836 

23592.52 J 

10008 

10000 

10002 

10 000 

30.0 

19964.85 ! 

29.9864 

19961.38 l 

29.9878 

29.9867 

19961.66 { 

10006 

10000 

10002 

10 000 

35.0 

17414.52 ! 

34.9888 

17411,50 J 

34.9900 

“34.9891 

17411.74 ! 

10004 

10000 

10002 

10000 

40.0 

1S545.61 J 

39.9907 

15542.91 J 

39.9916 

39.99C9 

15543.13 5 

10004 

ioooo 

10002 

10000 

45.0 

14135.39 ! 

44.9922 * 

14132.94 J 

44.9930 

44.9923 

14133.14 ! 

10003 

10000 

10002 

10000 

50.0 

13050.38 J 

49.9935 

13048.11 ! 

49.9941 

49.9936 

13048.29 ! 

10003 

10000 

1 0002 

10000 

60.0 

11546.45 1 

59.9956 

* 11544.45 ! 

59.9959 

59.9956 

11544.61 • 

1 0002 

10000 

10002 

100 00 

70.0 

10642.67 1 

69.9972 

10640.82 J 

69.9974 

69.9972 

10640.97 ! 

10002 

10000 

1 0002 

10 000 

80.0 

10155.76 i 

79,9987 

10154.00 \ 

79.9988 

79.9986 • 

10154.14 ; 

10002 

10000 

10002 

10 000 

90.0 

10001.69 J 

90.0000 

9999.95 J 

i 

90.0000 

90.0000 

10000.09 1 
» 

10002 

10000 

1 0002 

10000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 


COMPARISON OF GRADIENT (GMD) , UNITE SANDS (CYDER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
' BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


UEATHER DATA FOR EDUARDS EHA-7S HOT DAY SEA LEVEL INDEX OF REFRACTION l .0002939 SCALE HEIGHT! 7284.10 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB « 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD ! 

E MEAS 

R HEAS 

E COR 

R COR 

E COR 

E COR 

R COR 

NO CORR 

GMD H 

CYB H 

VAR H S 

DEG 

METER 

1 ' DEG 

METER 

DEG 

DEG 

METER 

METER 

METER 

METER 

METER 

0.0 

562319.17 

1 - . 4839 

S62220 .73 

-.6796 

-.3345 

562243.87 

24740 

20000 

18093 

21462 

.5 ‘ 

493781.09 

! .1072 

493703.71 

.0442 

.1943 

493712.38 

23380 

20000 

19465 

20749 

l .0 

436172.64 

.6729 

436108.53 

.6592 

.7359 

436115.49 I 

22487 

20000 

19901 

20478 

1.5 •' 

387496.89 

1.2221 

387443.00 

‘ 1.2291 

i .2697 

' 387449.07 

* 21877 

20000 

20051 

20321 

2.0 

346277.72 

1.7600 

346231.53 

1.7760 

i .7963 

346236.90 J 

21449 

20000 

20100 

20219 

2.5 

311302.40 

2.2898 

311262.17 

2.3096' 

- 2.3174 

311266.89 

21141 

20000 

20111 

20149 

3.0 

281542.48 

2.8138 

281506.97 

2.8349 

2,8349 

281511.08 , 

20914 

20000 

20107 

20104 

3.5 * 

256124.21 

3.3333 

256092.50 

3,3544 

3.3507 

256096,08 

20745 

20000 

20097 

20078 

4.0 

234313.18 

3.8494 

234284.60 

3.8700 

3.8660 

234287.72 l 

20615 

20000 

20087 

20068 

4;s * 

215498.33 

■ 4.3629 

215472.35 

4.3827 

*4 .*3817' 

215475.10 

20515 

20000 

20077 

20071 

5.0 

199176.12 

4.8744 

199152.32 

4.8932 

4.8929 

199154.77 i 

20437 

20000 

20068 

20064 

'6.0 * 

172435.20* 

5.8927 

172414.88 

Si90?6 

5'. 90 66 

*172416.87 

20323 

‘ ' 20000- 

20053 

20042 

7 . 0 

151615.91 

6.9066 

151598 . 20 

6.9218 

6.9176 

151599.92 

20247 

20000 

20043 

20029 

8.0 * 

135059.53 

7.9174 

135043.86 

7.9312 

‘ 7.9265 

135045.40 

20195 

20000 

20035 

20021 

9.0 

121640.52 

i 8.9261 

121626.48 i 

8.9386 ! 

8.9339 

121627.89 

20157 

20000 

20029 

20016 

io;o 

110580.07 

l 9.9333 

'110567.35 ! 

9.9447 

9.9400 

110568. 6S 

* 20129 

20000 

' 20024 

20013 

12.0 

93489.53 

! 11.9442 

93478.83 I 

11.9539 ! 

11.9495 

93479.97 \ 

20091 

20000 

200 18 

20009 

14'. 0 

80957 ", 45 

J 13.9523 

80948.21 1 

13*. 9605 

13.*9566 

80949.21 ! 

t 20068 

20000 

20014 

20006 

16.0 

71411 ,90 

I 15.9584 

71403.76 1 

15.9656 ! 

15.9620 

71404.66 1 

20052 

20000 

20011 

20 0 05 

18,0' 

63920.16 

! 17.9632 

63912.89 I 

17.9696' 

17.9663 

63913.69 

i 20041 

' 20000 

20009 

200 04 

20 . 0 

57897. 4S 

1 19.9671 

57890,87 I 

19.9728 I 

19.9699 

57891.61 

20033 

20000 

20008 

20 0 03 

25.0 

47038.42 

! 24.9743 

■ 47033.08 ! 

24,9788 

24.9764 

47033.69 

! * 20021 

20000 

20 0 06 

20 0 02 

30 . 0 

39843.33 

l 29,9793 

39838.82 ! 

29.9828 ! 

29.9809 

39839.33 

20015 

20000 

20 0 04 

20 001 

35 , 0 

34777.07 

! 34.9829 

34773.13 ! 

34.9858 

34.9842 

34773.58 

! 20011 

20000 

20 0 04 

20001 

40 . 0 

31058.33 

! 39.9858 

31054.82 ! 

39.9882 ! 

39.9868 

31055.22 

20008 

20000 

20 0 03 

20001 

45.0 

28249.23' 

! 44.9881 

' 28246,03 ! 

44,9901 

44.9889 

28246.40 

I 20006 

2000 0 

20 0 03 

20 00 1 
20000 
200 00 

50 . 0 

26086.22 

! 49.9900 

26083,26 j 

49.9917 ! 

49.9907 

26083.60 

! 20005 

20000 

20 0 03 

60 . 0 

23086, 18 

1 59.9931 

23083.57 

59.9943 

59.9936 

23083.87 

! 20004 

20000 

20 0 02 

70 . 0 

21282. 13 

! 69.9957 

21279.73 I 

69.9964 ! 

69.9960 

21280.00 

1 20003 

20 0 00 

20002 
20 0 02 
20002 

20000 
200 00 
20000 

80.0 

20309.96' 

! 79.9979 

20307.66 : 

79,9982 

79.9980 

20307.93 

* 20002 

20000 

90 . 0 

20002.26 

l 90.0000 
• 

1 

19999.99 I 

90.0000 ! 

1 

90.0000 

20000.25 

1 20002 
1 

1 

20000 













***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GHD), UNITE SANDS (CYDER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


UEATHER DATA FOR EDWARDS EHA-7S HOT DAY SEA LEVEL INDEX OF REFRACTION) .000293? SCALE HEIGHT) 7284.10 METERS 


INPUT 

DATA S 

GMD GRAD 

REF SOL 

1 

CYBER 

i SPHERICAL SLAB 2 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 

R MEAS 2 

E COR 

R COR 

2 

E COR 

2 E COR 

R COR 

NO CORR 

GMD H 

CYB H 2 

VAR H 

DEG 

HETER 2 

DEG 

METER 

2 

DEG 

2 * DEG 

METER 2 

METER 

METER 

METER 2 

METER 

0.0 

S 

867142.61 2 

- .5774 

867040 .SO 

i 

• 

1 

-1.0480 

> 

• -.4228 

* 

867066.77 2 

58676 

50000 

« 

42941 2 

52323 

* .S 

797350.90 ! 

.0262 

797269.91 

1 

-;i427 

I .1026 

797278.00 2 

56547 

50000 

47676 2 

51054 

1.0 

736085.33 1 

.6015 

736018.88 

2 

.5372 

2 .6594 

736025 . 42 2 

55083 

50000 

49187 2 

50737 

i.S 

681621.76 1 

1,1583 

681565.85 

2 

i . 1389 

2 1.2046 

681571.59 i 

54036 

50000 

49776 2 

50544 

2.0 * 

632873*. 46 1 

1 .7024 * 

63282S.47 

2 

1.7043 

2 1.7394 

632830.57 2 

53263 

50000 

50025 2 

S0404 

2.5 

589067.36 2 

2.2374 

589025.49 

i 

2.2499 

2 2.2666 

589029.97 2 

52679 

soooo 

50131 2 

50297 

- 3.0 

549603.95 S 

2.7657 

549566.93 

2 

2.7835 

2 2.7891 

* 549570.82 2 

52230 

50000 

50173 2 

50221 

3.5 

513990.52 i 

3.2890 

513957.42 

« 

i 

3.3093 

2 3.3092 

513960.78 2 

51878 

50000 

50184 2 

50179 

4.0 - 

481803.11 1 

3.8085 

481773.23 

< 

• 

- 3.8293 

2 3.8286 

481776.15 2 

51598 

SOOOO 

50181 2 

50168 

4.5 

452667.99 i 

4.3249 

452640.79 

2 

4.3463 

2 4.3484 

452643.35 i 

51372 

50000 

50171 2 

50184 

5.0 

426251.27 2 

4,8389 

426226.34 

! 

4.8600 

2 4.8621 

426228.59 2 

51188 ' 

* 50000 

50159 2 

50171 

6.0 

380438.62 2 

5.8615 

380417.29 

* 

5.8814 

I 5.8797 

380419.12 2 

50910 

50000 

50132 2 

50118 

7.0 ** 

342386.14 1 

6.8788 

342367.53 

• i 
* 

6.8972 

2 • 6; 8938 

342369.10 ? 

* 50715 

50000 

50109 2 

50087 

8,0 

310516.88 2 

7.8925 

310500.39 

* 

7.9094 

2 7.9052 

310501.81 2 

50574 

50000 2 

50091 2 

50066 

9.0 " 

283597.35 2 

' 8; 9036* 

283582.56 

\ 

8.9191 

2 8.9146 

* 283583.87 2 

50470 

50000 

50077 2 

50053 

10. 0 

260661.36 2 

9.9127 

260647.95 

» 

9.9270 

2 9.9225 

260649.18 2 

50391 

50000 2 

50066 2 

50043 

12.0 * 

223884*. 01 i 

11'. 9268 

223872.72 

i 

11.9390 

2* 11.9347 

* 223873.80* i 

50279 

50000 1 

1 50048 1 

50029 

14.0 

195916. S7 { 

13.9371 

195906.82 

• 

13.9478 

2 13.9438 

195907.78 2 

50203 

SOOOO 1 

50036 2 

50021 

16.0 

174075.64 2 

15.9451 * 

174067.05 

i 

* 

15.9545 

2 15.9508 

174067.91 2 

50160 

50000 ! 

50028 2 

50015 

18.0 

156626.66 2 

17.9514 

156618.98 

• 

i 

17.9598 

1 17.9564 

156619.76 I 

50126 

50000 ! 

50023 2 

50012 

* 20.0 

142415,83 ! 

19.9565 

142408.88 

* 

• 

19,9640 

2 19.9610 

142409.59 2 

50102 

50000 

50019 2 

50010 

25.0 

116413.75 J 

24.9660 

116408.10 

• 

• 

24.9719 

2 24.9694 

116408.69 1 

50064 

50000 ! 

50013 2 

50006 

30 .0 

98942.92 2 

29.9725 

98938,15 

i 

29.9772 

2 29.9752 

98938.65 2 

50043 

50000 

50009 2 

50004 

35 . 0 

86542.06 1 

34.9773 

86537.90 

i 

* 

34.9812 

2 34.9795 

86538.34 2 

50029 

50000 ! 

50006 2 

50002 

40.0 

77393.35 *i 

39.9810 

77389.64 

• 

i 

39.9843 

2 39.9829 

77390,03 2 

50021 

50000 

50005 2 

50001 

45.0 

70459.15 { 

44.9841 

70455.76 

* 

44.9868 

2 44.9857 

70456.12 2 

50015 

50000 ! 

50004 2 

50001 

50.0 

65107.27 1 

49.9867 

65104.15 

« 

i 

49.9890 

1 49.9880 

65104. 4B 2 

50011 

50000 

50003 2 

50000 

60.0 

57667.87 i 

59.9908 

57665.11 

. 

59.9924 

2 59.9917 

57665.41 2 

50007 

50000 ! 

50003 2 

50000 

70,0 

53185.76 2 

69.9942 

53183.21 

2 

69.9952 

2 69.9948 

53183.48 2 

50004 

50000 

50002 2 

50000 

80 . 0 

50768.00 2 

79.9972 

50765.57 

I 

79.9977 

2 79.9975 

50765.83 2 

50003 

50000 ! 

50002 2 

50000 

90 , 0 

50002.40 : 
1 
• 

90.0000 

50000.00 

* 

i 

i 

i 

90.0000 

2 90.0000 
1 
1 

50000.26 2 
* 

50002 

50000 

< 

50002 2 

50000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 











COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB ( VAR I AN ) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME HEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


' • WEATHER DATA FOR EDWARDS EHA-7S HOT DAY SEA LEVEL INDEX OF REFRACTION! ,0002939 SCALE HEIGHT! 7284.10 METERS 


INPUT 

DATA ' ' J 

GMD GRAD REF SOL ' 1 

CYDER \ 

■ SPHERICAL SLAB ! 

ALTITUDE 

COMPUTED BY 

1— . — r TT ■“ 11 “ 1 “■ ^ ™ 1 

EACH SOLUTION METHOD ! 

* E MEAS ‘ 
DEG 

R MEAS * ! 

METER ! 

E COR 
DEG 

R COR 
METER 


E COR 
DEG 

R COR 1 

METER ! 

NO CORR 
METER 

i GMD H ! 

METER l 

CYB H i 

METER J 

VAR H 
METER 

‘ * 0.0 ’ 

1 

1205849.49 1 

-.6266 

1205746.24 

I -1.4574 1 

-.4745 

« 

1205773.61 ! 

112988 

i 100000 

l 

1 82783 1 

103153 

.S 

1135461.54 ! 

-.0138 

1135379.90 

-.3019 ! 

.0475 

1135387.15 i 

11002? 

100000 

94387 J 

101194 

*1.0 

1072430.46 1 

* . S682 

1072363.61 

* .4470 l 

.6161 

1072369.64 \ 

107962 

100000 

97776 { 

100883 

i ,s 

1015085.41 1 

' i . 1299 

1015029.24 

i 1.0781' J 

' 1.1698 

1015034.68 S 

1 06459' 

1 100000 

99105 i 

100695 

2,0 

962424.17 ! 

1.6778 

962376.01 

1.6S92 1 

i .7106 

962380.91 \ 

105330 

100000 

99699 ! 

100542 

2 . S 

913802.01 5 

2.2158 

913760.02 

1 2.2142 I 

2.2422 ' 

913764.37 \ 

104464 

; looooo 

99982 J 

100415 

3.0 

868736.64 J 

2.7464 

968699.54 

2.7541 1 

2.7680 

868703.33 ! 

103785 

100000 

100119 : 

100321 

3.5 

826873.26 ! 

3.2717 ’ 

826840.10 

! 3.2842 1 

3.2907 

826843.40 \ 

103242 

! 100000 

100133 1 

100269 

4.0 

787933.94 ! 

3.7927 

787904.01 

3.8079 1 

3.8125 

787906.87 i 

102804 

100000 

100210 ! 

100266 

4.5 • 

751675.09 J 

4.3104 

751647.86’ 

[ 4.3269 ! 

* 4,3344 

•751650.38 ! 

102446 

100000 

100218 { 

100309 

5.0 

717874.88 1 

4.8255 

717849.92 

4.8426 ! 

4.8496 

717852.15 1 

102148 

100000 

100214 ! 

100296 

6.0 * 

656926.03 J 

5.8499 

656904.68 

1 S , 8668 ! 

*5.8690 

656906.48 ! 

101687 

100000 

100192 ! 

100213 

7.0 

603778.99 J 

6.8687 

6Q3760 .37 

6.8847 1 

6.8845 

603761.92 ! 

101355 

100000 

100167 ! 

100162 

8.0 * 

* 557306.31 J 

7.8835 

557289.81 

! 7.8984 ! 

* 7,8970 

557291.21 ! 

• 101109 

100000 

100145 1 

100129 

9.0 

516523,64 J 

8.8954 

516508.84 

8.9093 1 

8.9073 

516510.14 ! 

100919 

100000 

100124 ! 

100103 

■ '10,0 

480630.31* 1 

* 9.9053 

*480616.90 * 

i * 9.9182 ! 

’ 9.9158 

480618.11 ! 

100774 

100000 

100108 1 

100086 

12.0 

420749.16 i 

11.9205 

420737.87 

11.9317 1 

11,9291 

420738.94 1 

100S64 ] 

100000 

100080 1 

100060 

14. 0 

373244.80 ! 

13.9317 

373235.04 ' 

! 13.9415 J 

13.9390 

373235.99 ! 

' 100426 

100000 

100062 ! 

100044 

16,0 

334952.11 ! 

15.9403 

334943.52 

15.9490 ! 

15.9467 

334944.37 J 

100331 

100000 

100049 ! 

100034 

15 , 0 

303627.19 ! 

17.9472 

303619.51 

! 17.9549 J 

17.952 7 

**303620.28 ! 

' 100265 

i 0 0 0 0 0 

100041 1 

100028 

20 , 0 

277638.21 1 

19.9527 

277631.26 

19.9S97 ! 

19,9577 

277631.97 { 

100213 

1 0 0 0 Q 0 

100032 J 

1 0 0 021 

■ 25 ;o ' 

229063.48* \ 

*24.9629 

229057.84 

24.9684 i 

24.9663 ' 

229058.42 J 

* 100134 

100000 

100021 } 

100013 

30 , 0 

195740.42 ! 

29,9700 

195735.65 

29,9745 { 

29.9731 

195736.14 { 

100088 

100000 

100014 J 

10 0 0 08 

35 . 0 

171786.60 { 

34.9753 

171782.43 

'34.9789 i 

34.9778 * 

171782.87 S 

100061 

100000 

1 0 0 0 i 0 i 

10 0 0 05 

40 . 0 

153969.30 J 

39.9793 

153965.58 

39.9824 1 

39.981S 

153965.97 i 

100043 

100000 

ioooo8 : 

10 0 0 0 4 

45*. 0 

140389.65 ! 

44,9827 

140386.27 

44.9852 ! 

44.9844 

140386.63 ! 

' 100031 

100000 

100006 ! 

100002 

50 . 0 

129868.99 i 

49.9854 

129865.87 

49,9876 1 

49.9869 

129866.20 ! 

100023 

100000 

100006 ! 

100002 

60.0 

115188.29 J 

59.9900 

115185.52 

59.9915 ! 

59.9910 

115185.81 { 

100012 

100000 

100004 { 

100001 

70 , 0 

106315,75 S 

69.9937 

106313.21 

69.9946 J 

69.9943 

106313.48 J 

100006 

100000 

100003 ! 

10 0 0 0 0 

'80.0 

101521.62 ! 

79.9969 

101519.19 

79.9974 J 

79.9973 

101519. 4S ! 

100003 

100000 

100002 i 

100000 

90 , 0 
■ * * 

100002.40 : 
1 

90.0000 

100000.00 

90.0000 5 

i 

90.0000 

100000.26 ! 
1 
t 

100002 

100000 

I 

100002 1 

100000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS' OF THE SOLUTION METHOD 














APPENDIX E 


TABULATED REFRACTION COMPARISON DATA 
FOR EDWARDS EHC-75 COLD DAY CONDITIONS 



TABLE E-I 


WEATHER DATA FOR EDWARDS ECA-7S COLD DAY 


ALTITUDE 

(FT) 

TEMPERATURE 
(DEG C> 

DEW POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 1 0E6 

2316 

-.00000 

-4.15002 

939.00000 

289.57288 

3000 

-1.66439 

-5.02664 

909.01648 

281.44360 

4000 

-2.48001 

-5.65688 

894.95471 

277.35333 

5000 

-4.75101 

-7.73210 

857.09180 

265.91949 

6000 

-7.37825 

-10.15364 

813.58728 

252.78430 

7000 

-9.12493 

-11.82637 

785.50903 

244.31665 

8000 

-ii. 07649 

-13.87510 

756.26953 

235 . 54352 

9000 

-13.24860 

-16.24633 

724.61658 

226.08044 

10000 

-15.25633 

-18.48437 

695 . 53406 

217.41821 

11 0 00 

-17.23049 

-20.81900 

667,78772 

209.24750 

12000 

-19.27088 

-23.35292 

640.19519 

201.20953 

13000 

-21 .24707 

-25.85493 

614.24451 

193,69794 

14000 

-23.21263 

-28.41852 

589.68994 

186.66345 

15000 

-25.28627 

-31.27077 

565.09546 

179.71649 

16000 

-27.26981 

-34.09237 

541.60950 

173.10220 

17000 

-29.03263 

-36.68136 

519.72070 

166.89542 

18000 

-30.74469 

-39.27315 

497.69263 

160.63800 

19000 

-32.40592 

-41.69907 

476.33374 

154.58017 

20000 

-34.00380 

-43.74239 

456,59601 

149.00827 

21000 

-35.70151 

-45.38656 

437.04742 

143.52994 

22000 

-37.43376 

-46.81133 

417.84619 

138.15860 

23000 

-39.05200 

-48.19875 

399.96014 

133.12823 

24000 

-40.65479 

-49.71051 

382.71515 

128.21954 

25000 

-42.35757 

-51.29106 

365.55530 

123.30487 

26000 

-44.08086 

-52.80325 

349.32440 

118.66968 

27000 

-45.85164 

-54.22858 

334.10724 

114.35422 

28000 

-47.71517 

-55.75351 

318.86243 

109,98218 

29000 

-49.34718 

-57.32469 

304.25562 

105.67955 

30000 

-50.39213 

-58,91669 

290.78082 

101,48935 

31000 

-50.68689 

-60.63559 

277 . 44080 

96.95177 

32000 

-50.63933 

-62.21037 

264.53351 

92.38243 

33000 

-50.70688 

-63.30237 

252.63120 

88.22917 

34000 

-50.91084 

-63.58337 

241.11481 

84.28410 

35000 

-51.12187 

-63 . 36666 

229,87521 

80.43756 

36000 

-51.32311 

-63.15215 

219.38391 

76.84532 

37000 

-51.51958 

-63.15002 

209.48132 

73.44798 

38000 

-51.72641 

-63.15002 

199.71237 

70 . 09117 

39000 

-51.92905 

-63.15002 

190.47385 

66.91438 

40000 

-52.12324 

-63.15002 

181.91086 

63.96909 

41000 

-52.33182 

-63.15002 

173.44910 

61 . 05466 

42000 

-52.54131 

-63.15002 

165.34671 

58.26167 

43000 

-52.73788 

-63.15002 

157.88193 

55,68760 

44000 

-52.94424 

-63.15002 

150.57849 

53.16564 

45000 

-53.15488 

-63.15002 

143.50095 

50.71882 

46000 

-53.35290 

-63.15002 

136.94308 

48.45057 

47000 

-53.55122 

-63.15002 

130.65887 

46.27430 

48000 

-53.75803 

-63.15002 

124.49812 

44 . 13767 

49000 

-53,95930 

-63.15002 

118.71614 

42.13139 

50000 

-54.15699 

-63.15002 

113.31422 

40.25637 



TABLE E-I 


Continued 



WEATHER DATA FOR EDWARDS 

ECA-75 COLD DAY 

ALTITUDE 
(FT) * 

TEMPERATURE 
(DEG C) 

DEW POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 10E6 

51000 

-54.36859 

-63.15002 

107.97940 

38.40209 

52000 

-54.57560 

-63.15002 

102.90877 

36.63802 

53000 

-54.76559 

-63.15002 

98.22827 

35.00848 

54000 

-54.96605 

-63.15002 

93.61975 

33.40137 

55000 

-55.17589 

-63 . 15002 

89.18549 

31.85419 

56000 

-55.38883 

-63.15002 

85.09639 

30.42875 

57000 

-55.61987 

-63.15002 

81.13026 

29.04566 

58000 

-55 . 83823 

-63.15002 

77.27081 

27,69601 

59000 

-55.99423 

-63.15002 

73.67717 

26.43293 

60000 

-S6. 02567 

-63.15002 

70.28058 

25.22466 

61000 

-55.95256 

-63,15002 

66.94150 

24.02300 

62000 

-55.86035 

-63.15002 

63.79223 

22.88678 

63000 

-55.80771 

-63.15002 

60.87077 

21.83778 

64000 

-55.75486 

-63.15002 

57.99026 

20,80436 

65000 

-55.70129 

-63.15002 

55.24083 

19,81812 

66000 

-55.65079 

-63.15002 

52.70991 

18.91033 

67000 

-55.59756 

-63. 15002 

50.23737 

18.02381 

68000 

-55.54401 

-63.15002 

47.85069 

17.16836 

69000 

-55.49538 

-63.15002 

45.64375 

16.37751 

70000 

-55.44862 

-63.15002 

43.52708 

15.61938 

71000 

-55.39979 

-63. 15002 

41.46027 

14.87939 

72000 

-55.34921 

-63.15002 

39.52525 

14.18654 

73000 

-55.29395 

-63.15002 

37.71385 

13.53773 

74000 

-55.23518 

-63.15002 

35.92907 

12.89865 

75000 

-55.18587 

-63.15002 

34.23690 

12.29320 

76000 

-S5. 15603 

-63.15002 

32.67440 

11.73494 

77000 

-55,12713 

-63.15002 

31.13794 

11.18639 

78000 

-55.07240 

-63.15002 

29.66446 

' 10,65940 

79000 

-54.96227 

-63.15002 

28.30800 

10 . 17175 

80000 

-54.72646 

-63.15002 

26.99348 

9.69412 

81000 

-54.41663 

-63.15002 

25.71885 

9.22810 

82000 

-54.12528 

-63.15002 

24.53358 

8.79516 

83000 

-53.86273 

-63.15002 

23.40822 

8.38608 

84000 

-53.58611 

-63.15002 

22.30351 

7.98526 

85000 

-53.31669 

-63.15002 

21.26414 

7.60847 

86000 

-53.05879 

-63.15002 

20.30056 

7,25936 

87000 

-52.7838 7 

-63.15002 

19.35302 

6.91673 

88000 

-52.51028 

-63.15002 

18.45077 

6.59085 

89000 

-52.25340 

-63.15002 

17.61901 

6.29055 

90000 

-51.98223 

-63.15002 

16.80457 

5.99705 

91000 

-51.70617 

-63.15002 

16.01958 

5.71456 

92000 

-51.44562 

-63.15002 

15.29510 

5.45394 

93000 

-51.17860 

-63.15002 

14.59926 

5.20398 

94000 

-50.90123 

-63.15002 

13.92109 

4.96076 

95000 

-SO .63599 

-63.15002 

13.28648 

4.73335 

96000 

-50.37910 

-63.15002 

12.68924 

4.51955 

97000 

-50.10548 

-63.15002 

12.09976 

4.30894 

98000 

-49.83197 

-63.15002 

11.54172 

4.10969 

99000 

-49.56516 

-63.15002 

11 . 02721 

3.92596 

100000 

-49.28035 

-63.15002 

10.52198 

3,74586 



COMPARISON OF GRADIENT (GMD) , UNITE SANDS (CYDER) AND SPHERICAL SLAD ( VAR IAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR EDUARDS ECA-75 COLD DAY SEA LEVEL INDEX OF REFRACTION! .0003164 SCALE HEIGHT! 6891.33 METERS 


INPUT 

DATA 

GMD GRAD 

REF, SOL ! 

CYBER ! 

SPHERICAL SLAB \ 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD ! 

E MEAS 
DEG 

R MEAS 1 

METER * 

E COR 
DEG * * 

R COR s 

* METER * : 

E COR * 
DEG 1 

E COR 
DEG * * 

R COR \ 

METER * ! 

NO CORR : 
' METER * ! 


GMD H 
METER 


CYB H 
METER 

s 

UAR H i 

METER i 

0.0 

133569.05 

- . 1706 

J 

133S28 .49 \ 

i 

-.1877 ! 

-.1733 

* \ 
133521.16 ! 

i 

1398 : 


1000 

s 

961 


994 1 

.5 

76621.06 

.4040 * 

76598.21 [ 

; 3977 : 

.4005 

76596.68 : 

1129 i 


1000 


992 


995 ! 

1.0 

49392.03 

.9387 

49377.38 I 

.9350 S 

.9390 

49377.05 l 

1053 ! 


1000 


997 


looo : 

• i.S 

* 35480.70 

' 1.4563 

*35470.19 \ 

' 1.4535 ! 

* 1.4571 

35470.16 \ 

1027 ! 


1000 

1 

999 


1000 ! 

2.0 

27434.55 

1.9664 

27426.43 i 

1.9641 l 

1 .9667 

27426.47 ! 

1016 i 


1000 

. 

999 


1000 1 

2.5 

22283.21 

‘ 2.4729 * 

22276.62 ! 

2.4709 t 

'2.4726 

* 22276.6 7 i 

1011 1 


1000 

1 

1000 


looo i 

3.0 

18730.10 

2.9774 

18724.56 J 

2.9756 1 

2.9767 

18724.61 5 

1008 l 


1000 


1000 


1000 

3.5 • 

16141 . 11 

3.4806 

16136.34 i 

3.4790 ! 

3.4798 

16136.37 ! 

1006 


1000 

1 

1000 


1000 

4.0 

14174.81 

3.9831 

14170.62 l 

3.9815 S 

3.9825 

14170.65 ! 

1004 ! 


1000 


1000 


1000 

■ ' 4.5 

12632.59 

4.4850 

12628.86 i 

4 .'4836 S 

4.4850 

12628.88 J 

1004 


1000 

t 

t 

1000 


1000 

5.0 

11391.57 

4.9866 

11388.21 ! 

4.9852 1 

4.9860 

11388.22 S 

1003 


1000 

1 

1000 


1000 

6.0 ' 

'9519.86 

5.9890 

-* 9517.05 1 

5.9876 1 

5.9883 

9517.06 i 

1002 


1000 

i 

1000 


1000 

7.0 

8176.44 

6.9906 

8174.03 i 

6.9894 * 

6.9899 

8174.03 I 

1002 


1000 


1000 


1000 

8.0 

'7166. 10 

7.9919* 

7163.98 ! 

7.9907 i 

7,9912 

7163.99 i 

' 1001 


1000 

1 

1000 


1000 

9.0 

6379.32 

8.9929 

6377.43 ! 

8.9918 i 

8.9921 

6377.44 i 

1001 


1000 

\ 

1000 


1000 

1 10 . 0 

5749.45 

9.9937 * 

5747.75 I 

9.9926 ( 

9.9929 

5747 . 76 \ 

1001 


1000 

\ 

1000 


1000 

12.0 

4804.81 

11.9949 

4803.39 i 

11.9939 I 

11.9941 

4803.40 ! 

1001 


1000 

1 

1000 


1000 

t 14.0 

4130.78 

13.9958 

4129.56 ! 

13.9948 ! 

13.9950 

4129.56 i 

1001 


1000 


1000 


1000 

16.0 

3626.23 

15.9964 

3625.21 ! 

IS. 9955 ! 

15.9956 

3625.21 i 

1000 


1000 

1 

1000 


1000 

! 18.0 

3235,00 

17.9969 

3234.04 ! 

17.9960 ! 

17.9961 

■ 3234.04 1 

1000 


1000 

1 

1000 


1000 

20.0 

2923.24 

19.9973 

2922.38 i 

19.9964 \ 

19.9966 

2922.38 J 

1000 


1000 

1 

1000 


1000 

! ' 25.0 

2366.11 

24.9980 

2365.42 \ 

24.9972 ! 

24.9973 

2365.42 • 

1000 


1000 

1 

1 

1000 


1000 

! 30.0 

2000 . 02 

29.9985 

1999.43 1 

29.9977 ! 

29.9978 

1999.44 i 

1000 


1000 

< 

1000 


1000 

: 35.0 

1743.61 

34.9988 

* 1743.10 { 

34.9981 : 

34.9982 

1743.10 i 

1000 


1000 

1 

1 

1000 


1000 

: 40.0 

1555.98 

39.9991 

1555.52 ! 

39.9984 i 

39.9985 

1555.52 J 

1000 


1000 


1000 


1000 

! 45.0 

1414.52 

44.9993 

1414.10 \ 

44.9987 l 

44.9987 * 

* 1414.10 i 

1000 

1 

1 

1000 

1 

1 

1000 


10 00 

1 50.0 

1305.55 

49.9995 

1305.17 1 

49.9989 1 

49.9989 

1305.17 I 

1000 


1000 

1 

1000 


1000 

( 60.0 

1154.88 

59.9996 

*1154.54 1 

59.9992 : 

59.9993 

1154.54 J 

1000 

1 

1 

1000 

t 

1000 


1000 

: 70.0 

1064.37 

69.9998 

1064.06 l 

69.9995 *. 

69.9995 

1064.06 \ 

1000 


1000 

1 

1000 


1000 

S 80.0 

1 0 1 5 . 6*3 

79.9999* 

1015.33 ; 

79.9998 S 

79.9998 

1015.33 : 

1000 

1 

1000 

1 

1000 


1000 

1 90.0 

• * 

« 

1 

1000.20 j 

90.0000 
l * 

999.91 ! 

1 

90.0000 i 
1 

90 .0000 

999.91 i 

• * 1 

1 

1000 

1 

• 

1000 

1 

1 

1000 


1000 



***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL' ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR EDUARDS ECA-75 COLD DAY SEA LEVEL INDEX OF REFRACTION! .0003164 SCALE HEIGHT) 6891.33 METERS 


t 

1 

INPUT 

DATA I 

GMD GRAD 

REF SQL ! 

CYBER 

! SPHERICAL SLAB i 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

1 

\ 

l _ 

E MEAS 
DEG 

R MEAS ! 

METER ' 1 

E COR 
•DEG; 

R COR i 

METER t 

E COR 
DEG 

1 E COR 
! DEG ‘ 

R COR l 

’ ■ METER 'i 

NO CORR 
METER ‘ 

GMD H 
METER 

CYB H 
METER 

VAR H 
METER 

I 

1 

0.0 

s 

188252.45 i 

-.2363 

1 

188196.99 1 

-.2645 

I 

1 - . 237S 

i 

188194.27 l 

2778 

2000 

1909 

1996 

J 

.5 * 

125844 ‘.35 *1 

.3459 

125808.28 ! 

.3371 

\ .3394 

125806.44 i 

2339 

2000 

1981 

1986 

1 

1 

1.0 

88790.68 1 

.8925 

88765.50 ! 

.8886 

i .8945 

88765.27 ! 

2167 

2000 

1995 

2003 

• 1 - 

i.S 

* 66688.41 J 

1.4198 

66669.59 ! 

1.4175 

r 1.4229 

66669.84 1 

2094 

2000 

1998 

l 2004 

1 

1 

2.0 

52749.36 1 

1.9368 

52734.50 1 

1.9352 

\ i , 9389 

52734.86 1 

2059 

2000 

1999 

2002 

I 

2.5 

43386.50 1 

2.4482 

43374 .*30 1 

2.4469 

V 2.4492 

43374.64 I 

2040 

* 2000 

2000 

2001 

! 

3.0 

36743.93 ! 

2.9563 

36733.60 ! 

2.9551 

! 2.9564 

36733.90 ! 

£029 

2000 

2000 

2000 

: 

3.5 

31817.38 : 

3.4623 

31808.44 1 

3.4612 

! 3.4621 

31808.69 S 

2021 

2000 

2000 

2000 

i 

4.0 

28032.14 i 

3.9669 

28034 . 27 1 

3.9659 

S 3.9671 

28024.48 ! 

2017 

2000 

2000 

2000 

i 

•4.5 

£5039.11 » 

* 4.4706 

* 25032.08 I 

* 4.4696 

1 * 4.4717* 

• 25032 . 26 ! 

‘ £013 

2000 

2000 

2000 

i 

5.0 

22616.89 5 

4.9735 

22610.54 1 

4.9725 

i 4.9739 

22610.68 i 

2011 

2000 

2000 

2000 

M 5 

6.0 • 

18941.96 i 

5.9780 

18936.64 I 

5.9771 

! 5.9780 

18936.75 ! 

2008 

2000 

* * 2000 

2000 

1 s 

7.0 

16290.85 : 

6.9812 

16286.27 ! 

6.9803 

i 6.9810 

16286.37 i 

2006 

2000 

2000 

2000 

CJ1 ; 

8.0 

14290.87 1 

7.9837 

14236.85 ? 

7.9828 

S 7.9833 

' 14286.93 \ 

2005 

t 2000 

2000 

2000 

1 

1 

9.0 

12729.40 i 

8.9856 

12725.83 i 

8.9847 

i 8.9851 

12725.90 i 

2004 

2000 

2000 

2000 

1 

*10.0 

* 11477.61*1 

• 9.9871 * 

11474.39 ! 

9,9862* 

! 9.9866 

11474.45 i 

£003 

I • 2000 

2000 

2000 

1 

1 

12.0 

9597.26 1 

11.9894 

9594.57 \ 

11,9886 

! 11.9888 

9594.63 ! 

2002 

2000 

2000 ! 

2000 

1 

I 

’ 14.0 

8253.79 t 

13.9911 

8251.48 ! 

13.9903 

! 13.9905 

8251.53 ! 

2002 

1 2000 

2000 

2000 

t 

1 

16.0 

7247.40 J 

15.9923 

7245.37 i 

15.9915 

\ 15.9917 

7245.41 J 

2001 

2000 

2000 I 

2000 

1 

1 

‘18.0. ■ 

6466.58 ! 

17.9933 

6464.76 1 

17.9925 

i 17.9927 

6464.80 ! 

2001 

1 2000 

2000 

20 0 0 

1 

1 

20 . 0 

5843.88 i 

19.9941 

5842.24 S 

19.9933 

i 19.9934 

5842.27 l 

2001 

£000 

2000 j 

200 0 

1 

25.0 

4730.95 J 

24.9955 

4729.62 ! 

24.9948 

! 24.9949 

4729.65 ! 

2001 

2000 

2000 i 

20 0 0 

1 

1 

30.0 

3999.51 J 

29.9965 

3998.39 ! 

29.9958 

! 29.9959 

3998.41 i 

2001 

2000 

2000 ! 

20 0 0 

% 

1 

35.0 

3486.97 i 

34.9972 

* 3486.00 ! 

34.9965 

S 34.9966 

3486.02 I 

2001 

2C00 

2001 ! 

20 0 0 

1 

1 

40.0 

3111.67 : 

39.9977 

3110.80 : 

39.9971 

J 39.9971 

3110.82 » 

2001 

2000 

2000 ! 

20 0 0 

1 

45.0 

2828.84 I 

44.9981 

2828.05 ! 

44.9976 

: 44.9976 

2828.06 S 

2001 

2000 

2001 1 

20 0 0 

1 

50 . 0 

2611.17 ! 

49.9985 

2610.44 1 

49.9980 

S 49.9980 

2610.45 J 

2000 

2000 J 

20 0 0 ! 

20 0 0 

1 

1 

60.0 

• 2309.88 1 

59.9990 

2309.24 » 

59.9986 

i 59.9986 

2309.25 J 

2001 

2000 I 

200 0 ! 

20 0 0 

1 

70 . 0 

2128.89 ! 

69,9994 

2128.30 J 

69.9991 

i 69,9991 

2128.31 ! 

£001 

2000 

2001 ! 

20 0 0 

J 

eo.o 

2031,24 \ 

79,9997 

2030.67 l 

79.9996 

! 79.9996 

2030,68 I 

2000 

2000 

2000 ! 

20 0 0 

l 

• 

90.0 

2000,40 J 
1 

90.0000 

1999.83 i 
* 

90.0000 

i 90.0000 

i 

• 

1999.85 ! 

l 

2000 

2000 

2000 ! 

2000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), UHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR EDUARDS ECA-7S COLD DAY SEA LEVEL INDEX OF REFRACTION! .0003164 SCALE HEIGHT! 6891.33 METERS 


; INPUT 

DATA 

i GMD GRAD 

REF SOL 

! CYBER 1 

! SPHERICAL 

SLAB 

! ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD \ 

! E MEAS 

R MEAS 

! E COR 

R COR 

i E COR 1 

[ E COR 

R COR 

i NO CORR 

! GMD H 

1 CYB H ! 

VAR H : 

! DEG ‘ 

* METER 

S DEG 

* METER 

1 DEG 

:• DEG 

METER 

l METER 

! METER 

\ METER ! 

METER : 





1 

f 

0.0 

294816.03 * 

-.3515 

294737.10 

1 -.4142 

I -.2948 

294745.64 ! 

6810 1 

5000 

4680 

S291 


t 

1 

‘ .5 

227379.16 \ 

.2397 

227321.29 

i .2231 

! .2419 

227321.59 1 

6034 : 

5000 

4937 

5009 


1 

« 

1.0 

178832.12 ! 

.8000 

178787.87 

i .7977 

! .8075 

178789.08 1 

5626 ! 

5000 

4995 

S024 


1 

l-.s 

144178.37 i 

1.3409 

14414*3.26 

i* 1.3433 

T 1.3501 * 

144144.87 1 

5401 S 

5000 

5008 

5023 


1 

2.0 

119212.51 i 

1 .8696 

119183.76 

. 1.8734 

i 1.8771 

119185.37 ! 

5272 ! 

5000 

5009 

5016 


! 

2.5 

100846.80 I 

2.3904 

100822.61 

i * 2.3944 

\ 2.3954 

- 100824.06 ! 

5194 1 

5000 

5009 

5009 



3.0 

86991.96 1 

2.9058 

86971.18 

1 2.9098 

1 2.9090 

86972.43 ! 

5144 : 

5000 

5007 

5005 


• 

• 3.5 

76275.15 l 

3.4177 

76256.9 7 

: 3.4214 

i 3.4201 

’ 76258.03 S 

5111 i 

5000 

5006 

5003 


i 

4.0 

67792.75 ! 

3.9271 

67776.63 

: 3.9305 

: 3.9300 

67777.51 i 

5087 : 

5000 

5005 

5004 


! 

* 4.5 

' 60940.86 i 

4.4347 

’ 60926.38 

\ ' 4.4377 

: 4.4395 

60927.14 I 

5070 \ 

5000 

5004 

5005 


i 

■ 

5.0 

55306.65 ! 

4.9408 

55293.52 

l 4.9436 

l 4.9454 

55294.17 i 

5058 ! 

5000 

5004 

5004 

td 

i 

6.0 

' 46618.47 i 

5.9504 

46607.42 

l 5.9527 

* 5.9533 

46607.92 ! 

5041 ! 

5000 

5003 

5002 

1 

1 

7.0 

40255.19 ! 

6.9574 

40245.66 

i 6.9593 

1 6.9594 

40246.07 1 

5031 : 

5000 

5003 

5001 

C\ 

V 

8.0 

• 35407.19 S 

7.9627 

35398.81 

i 7.9644 

l 7.9641 

35399.17 l 

5024 ; 

5000 

5002 

5001 


i 

9.0 

31597.46 I 

8.9669 

31589.99 

i 8.9683 

! 8.9679 

31590.31 I 

5019 : 

5000 

5002 

5001 


! 

*10. 0 

28523.56 5 

9.9702 • 

28521.81 

i 9.9715 

i 9.9710 

28522.11 i 

5016 i 

5000 

5002 

5000 


> 

12.0 

23896.86 ' 

11.9753 

23891.21 

i 11.9763 

* 11.9758 

23B91.47 ! 

5011 ! 

5000 

5002 

5000 


i 

14.0 

20573.82 i 

13,9790 

20568.96 

1 13.9798 

: 13.9792 

20569.18 1 

5008 : 

5000 

5001 

5000 


i 

16.0 

18078,15 ! 

15.9818 

18073.87 

t 15.9824 

* 15.9819 

18074.07 I 

5007 ! 

5000 

5001 

5000 


• 

18.0 

16138.20 J 

17.9840 

16134.38 

I 17.9844 

: 17.9840 

16134.56 I 

5005 i 

5000 

5001 

5000 


• 

20.0 

14589.30 ! 

19.9BS8 

14585.85 

! 19.9861 

i 19.9857 

14586.02 S 

5005 t 

5000 

5001 

5000 


• 

» 

25.0 ' 

11817.34 ! 

24.9890 

11814.55 

\ 24.9892 

« 24.9888 

• 11814.68 l 

5003 i ‘ 

5000 

5001 

5000 


J 

30.0 

9993.19 ! 

29.9912 

9990.83 

\ 29,9912 

i 29.9909 

9990.94 J 

5002 \ 

5000 

5001 

5000 


, • 
i 

35.0 

8713.79 * 

34.9928 

8711.73 

\ 34.9928 

I 34.9925 

8711.83 i 

5002 S 

5000 

5001 

5000 


« 

40.0 

7777.03 i 

39.9941 

7775.20 

S 39.9940 

*. 39.9938 

7775.28 1 

5002 

5000 

5001 

5000 


■ 

45.0 

7070.53 1 

44.9951 

7068.86 

\ 44,9949 

i 44.9948 

7068.94 I 

5002 i 

5000 

5001 

5000 


1 

50 . 0 

6527.10 i 

49.9959 

6525.56 

S 49.9958 

: 49.9956 

6525.63 ! 

5001 ! 

5000 

5001 

5000 


< 

60.0 

5774.21 ! 

59.9972 

5772.85 

J 59.9971 

! 59.9970 

5772.92 1 

5001 : 

5000 

5001 

5000 


1 

70.0 

5321.83 t 

69.9983 

5320.57 

i 69.9982 

1 69.9981 

5320.63 \ 

5001 l 

5000 

5001 

5000 


■ 

80.0 

5078.23 1 

79.9992 

5077.03 

l 79.9991 

J 79.9991 

5077.09 i 

sooi : 

5000 

5001 

500 0 


i 

« __ 

90.0 

5001.14 i 

90 .0000 

4999.96 

i 90.0000 

! 90.0000 
. 

5000.01 J 
1 

sooi : 

i 

i 

5000 

5001 

5000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT <GMD>, WHITE SANDS (CYDER ) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 75-FM-6U 


M 

I 

-4 


WEATHER DATA FOR EDWARDS ECA-75 COLD DAY SEA LEVEL INDEX OF REFRACTION* .0003164 SCALE HEIGHT* 6891.33 METERS 


INPUT 

DATA 

1 GMD GRAD 

REF SOL 

S CYBER 

1 SPHERICAL 

SLAB 

1 ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 

R MEAS 1 

\ E COR 

R COR 

! E COR 

: E COR 

R COR 

! NO CORR I 

! GMD H 1 

\ CYB H “ S 

VAR H 

DEG 

METER 

1 DEG 

METER 

• DEG 

1 DEG 

METER 

l METER 

1 METER 

! METER ! 

METER 


i i i I * i I 


S 

0.0 

411426.01 I 

-.4535 

411330.33 l 

-.5781 

-.3328 

411349.95 1 

13256 * 

10000 1 

9112 

10866 

* 

i 

.s 

341565 . 48 \ 

.1453 

341491.90 1 

.1102 

.1984 

341497.49 1 

12115 \ 

10000 i 

9795 

10317 

1 

i.O 

286608.07 1 

.7140 

286549 .36 \ 

.7099 

.7507 

286553.99 J 

11431 i 

ioooo : 

9982 

10183 

* 

1.5 

• 243374.73 t 

1.2637 

243326.53 I 

1.2712 

1.2914 

243330.72 i 

11005 l 

ioooo : 

10034 

10118 

i 

2.0 

209297.67 I 

1.8006 

209257.14 S 

1.8124 

1.8213 

209260.89 » 

10729 i 

10000 l 

10045 

10076 

! 

‘2.5 

182269.10 l 

2.3287 

182234.35* 

2.3417 

2.3435 

182237.62 \ 

10546 i 

ioooo i 

10044 

10 047 

1 

3.0 

160621.38 ! 

2.8505 

160591.09 S 

2.8635 

2.8612 

160593.89 ! 

10420 S 

ioooo j 

10039 

10030 

J 

3.5 

143079.20 \ 

3.3678* 

143052.43 * 

3.3804 

3.3764 

143054.80 ! 

10331 : 

ioooo : 

10033 

10022 

1 

1 

4.0 

128686.18 i 

3.8818 

128662.24 ! 

3.8936 

3.8905 

128664.27 * 

10267 i 

ioooo ; 

10029 

10020 

{ 

' 4.5 

* 116730.38 1 

'4.3932 

116708.76 \ 

4.4044 

4.4044 

* 116710.51 i 

10219 i 

ioooo : 

10025 

10023 

I 

5.0 

106681.98 1 

4.9028 

106662,29 * 

4.9132 

4.9144 

106663.80 I 

10182 * 

ioooo : 

10021 

10022 

: 

6.0 ' 

• 90815.85 \ 

5.9178 

90799.16 ! 

5; 9269 

5.9260 

90800.36 i 

10131 ! 

ioooo i 

10016 

10013 

i 

7.0 

78925.53 \ 

6.9289 

78911.07 * 

6.9369 

6.9351 

78912.08 ! 

10099 : 

10000 J 

10013 

10009 

; 

• 8.0 

69725.40 \ 

* 7.9375 

69712. 6S ! 

7.9446 

7.9424 

69713.55 I 

10077 J 

ioooo ; 

10010 

10006 

• 

• 

9.0 

62417.14 \ 

8.9443 

62405.75 i 

Q.9S07 

8.9483 

62406.57 * 

10062 S 

10000 i 

10009 

10004 

i 

10.0 ' 

56483.61 1 

9.9499 

56473.31 I 

9.9556 

9.9532 

56474.06 ! 

10050 ! 

*10000 I 

10007 

• 10003 

• 

12.0 

47457.24 i 

11.9583 

47448.60 J 

11.9630 

11.9607 

47449.25 J 

10036 J 

ioooo * 

10006 

10002 

i 

14.0 

* 40934 .’67 I 

13.9644 

40927.22 i 

13.9684 

13.9663 • 

40927.79 ! 

10026 i 

ioooo : 

10005 

10001 

i 

16.0 

36013.58 ! 

15.9691 

36007.03 i 

15.9725 

15.9706 

36007.54 i 

10020 : 

ioooo i 

10004 

10001 

■ 

18.0 

32176.43 1 

17.9727 

32170.63 ! 

17.9757 

17.9739 

32171.09 i 

10016 ; 

ioooo ; 

10003 

1000 1 

* 

• 

20.0 

29106.06 ! 

19.9757 

29100.77 1 

19.9783 

19.9767 

29101.19 S 

10013 1 

ioooo : 

10003 

10001 

* 

25.0 

23598.26 i 

24 ; 981 1 ‘ 

23593,98 * 

24.9831 * 

24.9817 

23594.32 S 

10009 * 

10000 1 

10003 

10000 

i 

30.0 

19965.99 1 

29.9848 

19962.36 ! 

29.9863 

29.9852 

19962.66 ! 

10006 : 

10000 i 

10002 

IOOOO 

• 

i 

35.0 

17415.24 1 

' 34.9875 

17412.08 J 

34.9887 

l 34.9878 

17412.34 1 

1000S » 

ioooo i • 

10002 

IOOOO 

• 

40 . 0 

15546.10 I 

39.9896 

15543.28 i 

39.9906 

39.9898 

15543.50 l 

10004 ; 

ioooo * 

10002 

IOOOO 

* 

45.0 

14135.74 ! 

44.9913 

14133.17 I 

44.9921 

t 44.9915 

14133.38 l 

10003 * 

ioooo * 

10002 

IOOOO 

t 

• 

50.0 

13050.63 ! 

49.9927 

13048.26 i 

49.9934 

49.9928 

13048.45 i 

10003 ; 

ioooo ; 

10002 

10 0 0 0 

-i * 
* 

60.0 

11546.61 { 

59.9950 

11544.51 i 

59.9954 

! 59.9951 

11544.68 i 

10002 J 

ioooo : 

10002 

IOOOO 

• 

» 

70.0 

10642.78 ! 

69.9969 

10640.84 i 

69.9971 

69.9969 

10641.00 5 

100Q2 : 

ioooo : 

10002 

10000 

i 

* 

80.0 

10155. 8S l 

'79.9985 

10154.01 l 

79.9986 

! 79.9985 

10154.16 * 

10002 : 

ioooo * 

10002 

IOOOO 

! 

» 

i _ . 

90 . 0 

10001.77 i 
* 
i 

90.0000 

9999.96 1 
* 
• 

90.0000 ! 

1 

90.0000 

1 

lOQoo.io : 
• 

10002 : 
s 

ioooo : 
• 
• 

10002 

IOOOO 


i 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD) , UNITE SANDS (CYDER) AND SPHERICAL SLAD ( VARIAN) CORRECTION HETHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR EDUARDS ECA-75 COLD DAY SEA LEVEL INDEX OF REFRACTION i .0003164 SCALE HEIGHT i 6891.33 METERS 


INPUT 

DATA 

\ GMD GRAD 

REF SOL 

i CYBER 1 

t SPHERICAL 

SLAB 

i ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 

R MEAS 

l E COR 

R COR ■ 

S E COR 1 

t E COR 

R COR 

! NO CORR 1 

1 GMD H 1 

l CYB H « 

VAR H 

DEG 

METER 

1 DEG 

METER * 

! DEG 

i DEG 

METER 

i METER 

! METER 

1 METER ! 

METER 


I ! I I ! 


; o.o 

. 570798.50 1 

-.5522 

570692.98 

-.8020 

-.3987 

570720.25 1 

25490 t 

20000 : 

17527 1 

21527 

i .5 

499365.79 1 

‘ . 0562 

499283.34 

-.0245 

.1381 

499292.11 ! 

23862 1 

20000 : 

19306 : 

20713 

i 1.0 

439978.48 1 

.6331 

439911.61 

• 612S 

.6925 

439918.45 i 

22813 \ 

20000 i 

19848 1 

20455 

! 1.5 

390156.13 1 

‘ 1.1900 

390100.40 

• 1.1941 

* 1 . 2353 

* 390106.23 \ 

22108* \ 

20000 ! 

20033 I 

20308 

' 2.0 

348176.77 1 

1.7334 

348129.29 

1 .7482 

1.7681 

348134.33 S 

21618 i 

20000 : 

20094 { 

20210 

: 2.5 

312685.92 1 

2,2673 

312644.76 

2.2866 

2 .‘2934 

312649.07 1 

21268 1 

• 20000 ! 

20109 1 

20142 

: 3.0 

282569.46 1 

2.7943 

282533.25 

2.8153 

2.8142 

282536.91 ! 

21013 ! 

20000 : 

20106 ! 

20098 

i 3.5 

* 256900.24 ’1 

3.3162 

256868.00 

3,3374 

3.3327 

256871.09 i 

20823 J 

20000 ! 

20098 \ 

20074 

J 4.0 

234909.27 1 

3.8343 

234880.27 

3.8550 

3.8504 

234882.91 ! 

20679 . 

20000 : 

20087 J 

20066 

{ 4.5 

215963.26 1 

■ 4.3493 

215936.94 

4.3692 

4.3683 

215939.21 ! 

20567 ! 

20000 1 

20078 I 

20071 

J 5.0 

199543.98 i 

4.8620 

199519.91 

4 .8811 

4.8812 

199521.89 1 

20480 : 

20000 i 

20069 \ 

20066 

M } 6.0 

172674.58 ! 

5,8823 

172654.06 

5.8994 

5.8966 

172655,64 ! 

20354 ! 

20000 ! 

20054 : 

20043 

* 17.0 

151778.65 1 

6.8976 

151760.80 

6.9130 

6.9089 

151762. IS ! 

20271 1 

20000 i 

20043 ! 

20030 

00 : 8.o 

135174.45 1 

7.9096 

135158.66 

7.9235 

7.9188 

135159.88 1 

20213 :■ 

20000 i 

20035 J 

20022 

1 9.0 

121724.27 1 

8.9192 

121710.13 

8.9318 

8.9270 

121711.25 ! 

20171 ! 

20000 : 

20029 S 

20016 

1 10,0 

110642.73 1 

9.9270 

110629.93 

9.9385 

9.9337 

110630.97 ! 

20141 1 

20000 : 

20024 \ 

20013 

1 12.0 

93526.99 ! 

11.9390 

93516.22 

11 .9487 

11.9443 

93517.14 i 

20099 : 

20000 i 

20018 ! 

20009 

1 14.0 

30981.42 1 

13.9478 

80972.12 

13.9562 

13.9521 

80972.94 ! 

20074 ! 

20000 t 

20014 1 

20006 

1 16.0 

71428.06 1 

15.9545 

71419.89 

15.9618 

15.9581 

71420.61 ! 

20057 ! 

20000 ! 

20011 J 

20005 

S 18.0 

63931.52 ! 

17.9593 

63924.21 

17.9663 

17.9629 

63924.87 ! 

20045 i 

20000 { 

20009 : 

20003 

1 20.0 

57905.70 1 

19.9641 

57899.09 

19.9698 

19.9668 

57899.69 { 

20036 ; 

20000 1 

20008 i 

20003 

1 * 25.0 

47042.54 ! 

24.9720 

47037.18 

1 24.9764' 

! ' 24.9740 * 

47037.67 ! 

20023 i 

20000 i 

20006 : 

20002 

1 30.0 

39845.63 ! 

29.9774 

39841.09 

29.9809 

29.9789 

39841.51 { 

20016 ! 

20000 : 

20004 : 

20001 

1 35,0 

34778.44 1 

34.9814 

34774.48 

! 34,9843 

! 34.9826 

34774. 8S J 

20012 { 

20000 i 

£0004 ; 

2000 1 

1 40,0 

31059.19 ! 

39.9845 

31055.66 

39.9869 

39.9855 

31055.99 i 

20009 • 

20000 S 

20003 i 

2000 1 

! 45.0 

28249.78 ! 

44.9870 

28246.57 

I 44,9890 

! 44.9878 

28246.87 ! 

20007 J 

20000 i 

20003 1 

20 000 

: so.o 

26086.58 1 

49.9891 

26083.61 

49.9908 

49.9898 

26083.89 ! 

20005 t 

20000 : 

20003 : 

20000 

: 6o.o 

23086.34 1 

59.992S 

23083.71 

\ 59.9936 

59.9930 

23083.96 ! 

20004 S 

20000 ' ! 

20002 : 

20000 

1 70.0 

21282.20 ! 

69.9953 

21279.78 

! 69.9960 

69.9956 

21280.01 ! 

20003 ! 

20000 ! 

20002 : 

20000 

1 80.0 

20309.98 ! 

79.9977 

20307,68 

! 79.9981 

79.9978 

20307.89 1 

20002 1 

20000 I 

20002 J 

20000 

1 90.0 

• 

20002.27 ! 

90.0000 

19999.99 

: 90.0000 

i 

90.0000 

20000 .21 J 

20002 ; 
1 

20000 ; 

1 

20002 ! 

i 

20000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GhD) , WHITE SANDS (CYDER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR EDUARDS ECA-75 COLD DAY SEA LEVEL INDEX OF REFRACTION) .0003164 SCALE HEIGHT) 6891.33 METERS 


INPUT 

DATA 

\ GHD GRAD 

REF SOL 

CYBER 

SPHERICAL SLAB l 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E HEAS 

R MEAS 

\ E COR 

R COR 

E COR 

E COR 

R COR : 

HO CORR 

GMD H 

CYB H 

WAR H 

DEG 

METER 

I DEG 

METER 

* DEG ' 

: DEG 

HETER \ 

HETER 

METER 

METER 

METER 

0.0 

876249.10 

i -.6528 

876139.62 

1 • - 

-1.2312 

* 

-.4986 

i 

876170.42 l 

59909 

50000 

41230 

52341 

.5 

803553.69 

-.0295 

803468.34 

-.2314 

.0327 

803476.23 ! 

57372 

50000 

47198 

S0864 

i.O 

740485.32 

i .5586 

740416.13 

\ .4803 

i ' .6062 

740422.45 l 

55664 

50000 

47003 

50609 

1.5 

684838.99 

! 1.1241 

684781.28 

1.0975 

1.1629 

684786.79 1 

54461 

50000 

49690 

50459 

2.0 

635285.21- 

\ 1.6743 

635236.00 

1.6719 

1.7056 

635240.81 \ 

53584 

50000 

49977 

S0342 

2.5 

590914.02 

J 2.2139 

590871.29 

2.2233 

2.2383 

590875.43 ! 

52929 

50000 

50101 

50249 

•3.0 

551044,95 

i 2.7456 

551007.32 

t 2.7610 

2.7649 

551010.83 ! 

52428 

50000 

50151 

50183 

3.5 

515131.31 

1 3.2715 

515097.75 

3.2899 

3.2882 

515100.71 S 

52038 

50000 

50167 

50148 

4.0 

482720.05 

S 3.7930 

482689.82 

3.8126 

! 3.8105 

482692.33 I 

S1730 

50000 

\ 50167 

50146 

4.5 

453415.60 

1 4.3110 

453388.14 

4.3310 

4.3330 

453390.29 1 

51483 

50000 

50160 

50172 

■5.0 

426864.84 

! 4.8264 

426839,70 

4.8462 

i 4.8489 

’ 426841.57 i 

51282 

50000 

50149 

50166 

6.0 

380866. 11 

! 5.8510 

380844.64 

5.8698 

5 . 8685 

380846.12 i 

50979 

50000 

50125 

50113 

7.0 

342692.16 

{ 6.8699 

342673.46 

6.8872 

6.8840 

342674.74 : 

50768 

50000 

50103 

50082 

8.0 

310742.25 

J 7.8847 

310725.69 

7.9007 

7.8966 

310726. 8S ! 

50616 

50000 

50 086 

50062 

9.0 

283767.11 

! 8.8966 

283752.2 7 ! 

! 8.9113 

8.9070 

283753.35 \ 

50503 

50000 

50073 

50049 

10.0 

260791 .90 

i 9.9065 

260778.45 I 

9,9200 J 

9.9155 

260779.47 i 

50418 

50000 1 

50063 

50040 

12.0 

223965.70 

I 11.9216 

223954.38 1 

l 11.9332 

1 11.9289 

223955.29 ! 

50299 

50000 

50046 

50027 

14.0 

195970.26 

*. 13.9327 

195960.49 ! 

13.9429 

1 13.9389 

195961.30 • 

50222 

50000 

50034 

50019 

16.0 

174112.54 

! 15.9413 

174103.94 i 

15.9502 

! IS. 9466 

174104.67 i 

50171 

50000 

50027 

50014 

13.0 

156652.94 

I 17.9480 

156645.25 S 

17.9560 

17.9526 

156645.91 1 

50135 

50000 

50021 

50011 

20.0 

142435.10 

i 19.9535 

142428.14 I 

19.9606 

! 19.9576 

142428.75 \ 

50109 

50000 

50018 

50009 

25.0 

116423.56 

i 24.9636 

116417.91 ! 

24.9692 

24.9668 

116418.41 i 

50069 

50000 

50012 

50006 1 

• 30.0 

98943.41 

! 29.9706 • 

98943.64* 1 

29.9751 

1 29.9731 

98944 . 06 ! 

50045 

50000 

50003 

500 03 

3S . 0 

86545.35 

34.97S7 

86541.18 ! 

34.9795 

: 34.9778 

86541.56 J 

50031 

50000 

50006 

50002 

40 . 0 

77395.41 

i 39.9797 

77391.69 i 

39.9828 

! 39.9814 

77392.02 S 

50022 

50000 

50005 

500 01 

45.0 

70460.47 

! 44.9830 

70457.08 ! 

44.9856 

44.9344 

70457.39 J 

50016 

50000 

500 04 

50001 

50.0 

65103.13 

I 49.9857 

65105.01 1 

1 49.9879 

1 49.9869 

65105,29 J 

50012 

50000 

50003 

50000 

60 . 0 

57668.24 

\ 59.9902 

57665.47 i 

59.9917 

59.9910 

57665.72 ! 

50007 

50000 

50003 

50000 

• 70.0 

53185.89 

69.9938 

53183.34 I 

1 69.9948 

! 69.9943 

53183.58 J 

50004 

50000 

50002 

50 0 00 

80 . 0 

50768.03 

J 79.9970 

50765.60 J 

79.9975 

79.9972 

50765.82 ! 

50003 

50000 

50002 

50 00 0 

90 . 0 

S00 02 . 4 0 

J 90.0000 * 

i 

50000.00 ! 

1 

1 90.0000 

t 90 .0000 

50000.22 \ 

i 

• 

50002 

50000 

50002 

I 

50000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYDER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


' WEATHER DATA FOR EDUARDS ECA-7S COLD DAY SEA LEVEL INDEX OF REFRACTION) .0003164 SCALE HEIGHT: 6091.33 METERS 


INPUT 

DATA 

1 GMD GRAD 

REF SOL 

S CYBER 

i SPHERICAL 

SLAB 

! ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS 

R MEAS 

l E COR 

R COR 

i E COR 

l E COR 

R COR 

S NO CQRR ! 

l GMD H ! 

! CYB H J 

VAR H 

DEG 

METER 

i DEG 

METER 

: DEG 

1 DEG 

METER 

1 METER ! 

METER { 

METER ! 

METER 


■ i - 

0.0 

1215254.84 

l 

-.7059 

1215144 . 0.1 

1 

-1.7075 

i 

-.5557 

1215176.10 i 

114741 1 

I 100000 

79072 

103138 

* 

.5 

1141960 *34 

l 

-.0719 

1141874.24 

\ 

-.4052 

I 

-.0282 

1141881.92 1 

111225 \ 

100000 

93469 

100856 


1.0 

1077122.28 

\ 

.5237 

1077052.65 

» 

■ 

.3842 

• 

* 

.5591 

1077058.35 \ 

108820 1 

100000 

97428 

100655 

: 

i.s 

1018588.46 

i 

i 

i . 0947 

1018530.47 

1 

1.0335 

\ 

1.1256 

1018535.62 1 

107098 1 

: iooooo 

[ 98939 

100541 

i 

2.0 

965108.67 

i 

i 

1.6491 

965059.28 

* 

* 

1.6247 

i 

1.6751 

965063.88 ! 

105821 

100000 

99603 

100430 

! 

2.5 

915912.14 

\ 

2.1917 

915869.29 

1 

2.1862 

» 

• 

2.2126 

915873.30 i 

104853 

: iooooo 

1 99920 

100328 

i 

3.0 

870423.64 

t 

* 

2.7260 

870385.92 

• 

* 

2.7305 

* 

* 

2.7428 

870389.33 l 

104098 

iooooo 

100073 

100250 

s 

3.5 

828247.23 

t 

3.2539 

828213.61 

: 

3.2639 

* 

• 

3.2690 

828216.50 1 

103500 

^ iooooo 

1 100147 

100213 

1 

4.0 

789069.01 

\ 

3.7770 

789038.73 

i 

3.7900 

i 

• 

3.7938 

789041.19 i 

103020 

iooooo 

100180 

100226 

1 

1 

4.5 

752621.29 

\ 

4.2964 

752593.79 

i 

4.3110 

J 

4.3185 

752595.91 S 

102629 

! 100000 

1 100192 

100284 

1 

1 

5.0 

718674.07 

l 

4.8130 

718648.90 

* 

4.8282 

: 

4.8361 

718650.74 : 

102305 

IOOOOO 

100192 

100284 

\ 

6.0 

657509 . 24 

l 

5.839S 

657487.76 

• 

* 

5.8548 

• 

» 

5 T8575 

657489.21 ! 

101806 

! 100000 

! 10017S 

100201 

t 

« 

7.0 

604216.74 

\ 

6.8597 

604198.02 

\ 

6.8744 

» 

6.8745 

604199.28 ! 

101448 

IOOOOO 

100154 

100151 

5 

8.0 

557641.11 

\ 

7.8756 

557624.55 

\ 

7.8894 


7.8882 

557625.69 l 

101183 

: iooooo 

i 100134 

100119 

« 

« 

9.0 

516785.77 

S 

8.8885 

516770.92 

i 

8.9013 

i 

8.8995 

516771.99 S 

100980 

iooooo 

100114 

100095 

• 

iO.O 

480837.62 

\ 

9.8990 

480824.17 

\ 

9.9110 

i 

9.9088 

480825.18 1 

100824 

: iooooo 

! 100100 

100079 

1 

12.0 

420884.96 

i 

11.9153 

420873.64 

i 

11.9257 

• 

« 

11.9233 

420874.54 i 

100600 

iooooo 

100074 

100055 

1 

1 

' 14.0 

373337.36 

* 

» 

13.9273 

373327.59 

i 

13.9364 

* 

• 

13.9340 

373328.39 J 

100454 

: iooooo 

! 100058 

100040 

ft 

16.0 

335017.57 

« 

* 

15.9365 

335008.97 

i 

15.9445 

• 

• 

15.9423 

335009.69 l 

100352 

iooooo 

I 100045 

100030 

ft 

1 

18.0 

303674.71 

s 

17.9438 

303667.01 

i 

17.9510 

* 

* 

17.9489 

303667.67 S 

100282 

; iooooo 

: 100038 

100025 

1 

20.0 

277673.72 

i 

% 

19:9497 

277666.76 

* 

• 

19.9562 

• 

» 

19.9542 

277667.36 ! 

100227 

1 IOOOOO 

! 100030 

100019 

1 

25.0 

229081.96 

i 

* 

24.9606 

•229076.31 

* 

* 

24.9657 

i 

24.9641 

229076.81 : 

100142 

! 100000’ 

! 100020 

100012 

« 

30.0 

195750.95 

* 

* 

29.9681 

195746.17 

* 

» 

29.9723 

» 

* 

29.9709 

195746.59 ! 

100094 

! 100000 

! 100013 

100007 

1 

1 

35.0 

171792.95 

• 

34.9737 

171788.79 

> 

• 

34.9771 

i 

* 

34.9760 

171789.16 S 

100065 

! 100000- 

: ioooio 

100005 

1 

1 

40.0 

1S3973 .30 

• 

39.9780 

1S3969 .58 

• 

• 

39.9809 

* 

* 

39.9800 

153969.91 ; 

100046 

I IOOOOO 

; 100007 

100003 

1 

1 

45.0 

140392.24 

i 

■ 

44.9816 

140388.85 

■ 

* 

44.9840 

* 

» 

44.9832 

• 140389.16 { 

100033 

\ IOOOOO 

: 100006 

100002 

1 

ft 

50.0 

129870.68 

■ 

• 

49.9845 

129867.56 

* 

• 

49.9865 

» 

• 

49.9859 

129867.84 ! 

100025 

i IOOOOO 

! 100005 

100002 

1 

1 

60.0 

115189.00 

* 

59.9894 

115186.24 

» 

* 

59.9907 

• 

• 

59.9903 

115186.48 ! 

100013 

; iooooo 

: 100003 

100001 

ft 

70.0 

106316.02 

i 

• 

69.9933 

106313.47 

* 

• 

69.9942 

• 

* 

69.9939 

106313.70 j 

100006 

: iooooo 

! 100003 

iooooo 

1 

80 . 0 

101521.68 

* 

• 

79.9967 

101519.25 

* 

* 

79.9972 

« 

* 

79.9970 

101519 . 47 l 

100003 

: iooooo 

; ioooo 2 

IOOOOO 

1 

t 

a 

90.0 

100002.40 

i 

* 

• 

• 

90.0000 

100000.00 

* 

• 

» 

* 

90 .0000 

i 

- < 
* 

90.0000 

100000.22 : 
• 
• 

100002 : 

I IOOOOO 

ft 

1 

1 100002 
1 

iooooo 

1 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 


APPENDIX F 

TABULATED REFRACTION COMPARISON DATA 
FOR EDWARDS TYPICAL COLD MOIST MORNING 


F-l 



TABLE F-I 


WEATHER DATA FOR TYPICAL COLD-MOIST MORNING 


ALTITUDE 

<FT) 

TEMPERATURE 
(DEG C) 

DEW POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 10E6 

2316 

-7. 15002 

-7. 15002 

939.00000 

293.07935 

3000 

-3.74665 

-8.05864 

909.01648 

279.30591 

4000 

-3.21093 

-10.03486 

894.95471 

273. 11621 

5000 

-3.71S09 

-17.29132 

857.09180 

257.53223 

6000 

-4.57112 

-25.27481 

813.58728 

240.18613 

7000 

-5.8S618 

-29.43772 

785.S0903 

230.49646 

8000 

-8.76741 

-31.32787 

756.26953 

223.78452 

9000 

-12.37808 

-32.37266 

724.61658 

217.63535 

10000 

-15.34153 

-33.30928 

695.53406 

211 . 4853S 

11000 

-17.51992 

-34.02937 

667.78772 

204.74118 

12000 

-19.42188 

-34.49619 

640.19519 

197.72952 

13000 

-21.24974 

-35.05724 

614.24451 

191.08826 

14000 

-23.21263 

-36.03225 

S89. 68994 

184.83401 

15000 

-25.28627 

-37.34002 

565.09546 

178.45306 

16000 

-27.26981 

-38.67367 

541.60950 

172.28491 

17000 

-29.03263 

-39.82956 

519.72070 

166.45926 

18000 

-30.74469 

-41.04211 

497.69263 

160.42966 

19000 

-32.40592 

-42.29614 

476.33374 

154.50446 

20000 

-34.00380 

-43.55772 

456.59601 

149.03824 

21000 

-35.70151 

-45.04784 

437.04742 

143,58487 

22000 

-37.43376 

-46 . 66737 

417.84619 

138.18195 

23000 

-39.05200 

-48.19875 

399.96014 

133.12823 

24000 

-40.65479 

-49.71051 

382.71515 

128.21954 

25000 

-42.35757 

-51.29106 

365.55530 

123.30487 

26000 

-44.08086 

-52.80325 

349.32440 

118.66968 

27000 

-45.85164 

-54.22858 

334.10724 

114.35422 

28000 

-47.71517 

-55.75351 

318.86243 

109.98218 

29000 

-49.34718 

-57.32469 

304.25562 

105.67955 

30000 

-50.39213 

-58.91669 

290.78082 

101.48935 

31000 

-50.68689 

-60.63559 

277.44080 

96.95177 

32000 

-50.63933 

-62.21037 

264.53351 

92.38243 

33000 

-50.70688 

-63.30237 

252.63120 

88.22917 

34000 

-50.91084 

-63.58337 

241.11481 

84.28410 

3S000 

-51,12187 

-63.36666 

229.87521 

80.43756 

36000 

-51.32311 

-63.15215 

219.38391 

76.84532 

37000 

-51.51958 

-63.15002 

209.48132 

73.44798 

38000 

-51.72641 

-63.15002 

199.71237 

70.0911 7 

39000 

-51.92905 

-63.15002 

190.47335 

66.91438 

40000 

-52.12324 

-63.15002 

181.91086 

63.96909 

41000 

-52 . 33182 

-63.15002 

173.44910 

61.05466 

42000 

-52.54131 

-63.15002 

165.34671 

58.26167 

43000 

-52.73788 

-63.15002 

157.88193 

55.68760 

44000 

-52.94424 

-63.15002 

150.57849 

53.16564 

45000 

-53. 15488 

-63.15002 

143.50095 

50.71882 

46000 

-53.35290 

-63.15002 

136.94308 

48.4S057 

47000 

-53.55122 

-63.15002 

130.65887 

46,27430 

48000 

-53.75803 

-63.15002 

124.49812 

44. 13767 

49000 

-53.95930 

-63.15002 

118.71614 

42.13139 

50000 

-54.15699 

-63.15002 

113.31422 

40.25637 
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TABLE F-I 


Continued 


WEATHER DATA FOR TYPICAL COLD-MOIST MORNING 


ALTITUDE 

<FT) 

TEMPERATURE 
(DEC C) 

DEU POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 1 0E6 

2316 

-7.15002 

-7.15002 

939.00000 

293.07935 

3000 

-3.74665 

-8.0S864 

909.01648 

279.30591 

4000 

-3.21093 

-10.03486 

894.95471 

273,11621 

5000 

-3.71509 

-17.29132 

857.09180 

257.53223 

6000 

-4.57112 

-25.27481 

813.58728 

240.18613 

7000 

-5.85618 

-29.43772 

785.50903 

230.49646 

8000 

-8.76741 

-31.32787 

756.26953 

223.78452 

9000 

-12.37808 

-32.37266 

724.61658 

217.63535 

10000 

-15.34153 

-33.30928 

695.53406 

211.48535 

11000 

-17.51992 

-34.02937 

667.78772 

204.74118 

12000 

-19.42188 

-34.49619 

640.19519 

197.72952 

13000 

-21.24974 

-35.05724 

614.24451 

191.08826 

14000 

-23.21263 

-36.03225 

589.68994 

184.83401 

15000 

-25.28627 

-37.34002 

565.09546 

178,45306 

16000 

-27.26981 

-38.67367 

541.60950 

172.28491 

17000 

-29.03263 

-39.82956 

519.72070 

166.45926 

18000 

-30.74469 

-41.04211 

497.69263 

160.42966 

19000 

-32.40592 

-42.29614 

476.33374 

154.50446 

20000 

-34.00380 

-43.55772 

456.59601 

149.03824 

21000 

-35.70151 

-45.04784 

437.04742 

143.58487 

22000 

-37.43376 

-46.66737 

417.84619 

138.18195 

23000 

-39. 0S200 

-48.19875 

399.96014 

133.12823 

24000 

-40.65479 

-49.71051 

382.71515 

128.21954 

25000 

-42.35757 

-51.29106 

365.55530 

123.30487 

26000 

-44.08086 

-52.80325 

349.32440 

118.66968 

27000 

-45.85164 

-54.22858 

334.10724 

114.35422 

28000 

-47.71517 

-55.75351 

318,86243 

109.98218 

29000 

-49.34718 

-57.32469 

304.25562 

105.67955 

30000 

-50.39213 

-58.91669 

290.78082 

101.48935 

31000 

-50.68689 

-60.63559 

277.44080 

96.95177 

32000 

-50.63933 

-62.21037 

264.53351 

92.38243 

33000 

-50.70688 

-63.30237 

252.63120 

88.22917 

34000 

-50,91084 

-63.58337 

241 .11481 

84.28410 

35000 

-51,12187 

-63.36666 

229.87521 

80.43756 

36000 

-51.32311 

-63.15215 

219.38391 

76.84532 

37000 

-51.51958 

-63.15002 

209.48132 

73.44798 

38000 

-51.72641 

-63.15002 

199.71237 

70.0911 7 

39000 

-51.92905 

-63.15002 

190.47335 

66.91438 

40000 

-52.12324 

-63.15002 

181.91086 

63.96909 

41000 

-52.33182 

-63.15002 

173.44910 

61.05466 

42000 

-52,54131 

-63.15002 

165.34671 

58.26167 

43000 

-52.73788 

-63.15002 

157.88193 

55.68760 

44000 

-52.94424 

-63.15002 

150.57849 

53. 16564 

45000 

-53.15488 

-63.15002 

143.50095 

50,71882 

46000 

-53.35290 

-63.15002 

136.94308 

48.45057 

47000 

-53,55122 

-63.15002 

130.65887 

46.27430 

48000 

-53.75803 

-63.15002 

124.49812 

44 . 13767 

49000 

-53.95930 

-63.15002 

118.71614 

42.13139 

50000 

-54.15699 

-63.15002 

113.31422 

40.25637 
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COMPARISON OF GRADIENT (GMD> , WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL COLD-MOIST MORNING SEA LEVEL INDEX OF REFRACTION: .0003226 SCALE HEIGHT: 6779.43 METERS 


INPUT 

DATA 

t GMD GRAD 

REF SOL 

\ CYBER ! 

SPHERICAL 

SLAB 

i ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E HEAS 

R HEAS 

1 E COR 

R COR 

1 E COR i 

E COR 

R COR 

i NO CORR J 

! GMD H ! 

I CYB H i 

VAR H 

DEG 

METER 

J ' * DEG 

METER 

S DEG S 

! DEG 

METER 

! METER 

J METER 

1 METER 

METER 


0.0 

134527 . 58 i 

- . 1780 

134486.06 

1 

- . 1966 S 

- . 1808 

134478.67 ! 

1419 S 

1000 

9S7 

993 

.5 

76880.32 \ 

.4003 

76BS7.08 

1 

.3933 

.3966 

76855.35 S 

1134 i 

1000 

991 

995 

1.0 

49471.50 i 

.9365 

49456.63 

1 

.9323 

.9367 

49456.21 i 

1055 i 

1000 

997 

1000 

1.5 

35511 .63*1 

1.4547 

*35500.98 

\ 

1.4517 

1 , 4S55 

35500.89 \ 

1028 i 

1000 

998 

1000 

2.0 

27449.16 ! 

1.9652 

27440.94 

i 

• 

1.9627 

1.96S5 

27440,93 i 

1017 i 

1000 

999 

1000 

2.5 

22291.13 S 

2.4720 

22284.45 

: 

2.4693 

2.4716 

22284.46 J 

1011 : 

1000 

999 

1000 

3.0 

18734.83 1 

2.9766 

18729.22 

: 

2.9746 ! 

2.9758 

18729.23 S 

1008 ; 

1000 

1000 

1000 

3.5 

16144.15 ! 

3.4799 

16139.31 

i 

3.4782 

3.4791 

16139.32 J 

1006 i 

1000 

1000 

1000 

4.0 

14176.8 7 \ 

3.9825 

14172.63 

i 

3.9808 i 

3.9819 

14172.63 I 

1005 : 

1000 

1000 

1000 

4.5 

12634.05 S 

4.4845 

12630.27 

J 

4.4829 

' 4.4845 

12630.27 J 

1004 i 

1000 

1000 

1000 

5.0 

11392.65 ' 

4.9861 

11389.24 

* 

• 

4.9846 

4.9855 

11389.24 J 

1003 i 

1000 

1000 

1000 

6.0 

- 9520.49 * 

5.9886 

9517,64 

t 

S .9872 

5.9879 

9517.64 J 

1002 i 

1000 

1000 

1000 

7,0 

8176.85 S 

6.9903 

8174.40 

« 

6.9890 

6.9896 

8174.39 S 

ioo2 : 

1000 

1000 

1000 

8.0 

7166.37 i 

7.9916 

7164.23 

* 

7.9904 

7.9908 

7164.22 1 

1001 \ 

1000 

1000 

1000 

9.0 

6379.51 J 

8.9927 

6377.61 

i 

8.991S 

8.9919 

6377.60 ! 

iooi i 

1000 

1000 

1000 

iO'.O 

5749.60 \ 

9.9935 

5747.88 

\ 

9.9923 

9.9927 

‘ 5747.87 i 

1001 J 

1000 . 

1000 

1000 

12.0 

4804.90 * 

11.9947 

4803.46 

i 

11.9936 * 

11.9939 

4803.46 i 

1001 \ 

1000 

1000 

1000 

14,0 

4130.84 ! 

13.9956 

4129.60 

* 

» 

13.9946 ! 

1 13.9948 

4129.60 ! 

iooi i 

1000 

1000 

1000 

16.0 

3626.32 i 

15.9963 

3625.23 

i 

• 

15.9953 ‘ 

15.9955 

3625,23 i 

1001 s 

1000 

1000 

1000 

18.0 

3235.03 ! 

17.9968 

3234.06 

* 

• 

17 . 9958 ! 

! 17.9960 

3234.06 • 

1000 i 

1000 

1000 

1000 

20.0 

2923.27 i 

19.9972 

2922.39 

* 

* 

19.9963 i 

19.9964 

2922.39 i 

1000 [ 

1000 

1000 

1000 

25.0 

2366.13 i 

24.9980 

2365.42 

j 

24.9971 1 

1 24.9972 

2365.42 \ 

1000 \ 

1000 

1000 

1000 

30.0 

2000.04 t 

29,9985 

1999.44 

* 

* 

29.9977 { 

29.9978 

1999.44 ! 

1000 i 

1000 

IOOO 

10 0 0 

35 . 0 

1743.62 i 

34.9988 

1743.10 

i 

34.9981 ! 

34.9981 

1743.10 i 

1000 i 

1000 

1000 

10 00 

40 . 0 

1555.99 ! 

39.9991 

1S55.52 

: 

39.9984 S 

39.9985 

1555.52 1 

iooo : 

1000 

1000 

10 00 

45. 0 

1414.53 ! 

44.9993 

1414.10 

» 

« 

44.9986 1 

44.9987 

1414.10 J 

1000 J 

IOOO 

1000 

10 0 0 

50 . 0 

1305.56 I 

49.9995 

1305.17 

i 

« 

49.9989 I 

49.9989 

1305.16 I 

iooo : 

1000 

1 00 0 ! 

10 0 0 

60 . 0 

1154.88 i 

59.9996 

1154.54 

• 

* 

59.9992 ! 

59,9992 

1154.54 i 

iooo : 

1000 

1000 

10 00 

70.0 

1064.38 S 

69.9998 

1064.06 

• 

■ 

69.9995 ! 

69.9995 

1064.06 S 

iooo : 

1000 

1000 

10 0 0 

80 . 0 

1 0 15 . 63 I 

79.9999 

1015.33 

i 

• 

79.9998 

79.9998 

1015.33 5 

1000 1 

1000 

1000 

10 0 0 

90 . 0 

1000.21 : 

i 

90.0000 

999.91 

j 

i 

♦ 

90.0000 1 

90.0000 

999.91 ! 

< 

iooo ; 
« 

1000 

1000 

• 

1000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 
WEATHER DATA FOR TYPICAL COLD-MOIST MORNING SEA LEVEL INDEX OF REFRACTION i .0003226 SCALE HEIGHT! 6779.43 METERS 


\ INPUT 

DATA 

GMD GRAD 

REF SOU 

CYBER 

SPHERICAL SLAB \ 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

1 E ME AS 

R hEAS 

E COR 

R COR 

E COR 

E COR 

R COR 

NO CORR 

GMD H 

CYB H- 

VAR H 

I DEG * 

i 

METER 

DEG 

METER 

DEG 

' DEG 

1 METER i 

METER 

METER 

METER 

METER 

i 

i 

1 0.0 

189399.58 

1 

-.2452 

189343.59 

-.2768 

-.2474 

1 

189340.33 1 

2811 

2000 

1897 

1993 

! • .5 

126237.21 

.3413 

126201.15 

.3303 

.3332 

* 126198.42 S 

2350 

2000 

1977 

1982 

{ 1,0 

88936.81 

.8897 

88911. 7S 

.8843 

.8906 

88910.88 ! 

2172 

2000 

1992 

2001 

i i.S 

66751.75 

1.4178 

66733.06 

1.4143 

1.4201 

66732.83 1 

2096 

2000 

1997 

2003 

i 2.0 

52781.07 

1.9354 

52766.34 

1.9327 

1 .9368 

52766.31 S 

2060 

2000 

1998 

2001 

l 2 . S 

43404.23 

2.4471 

43392.13 

2.4449 

! 2.4474 

43392.15 S 

2041 

2000 

1999 

2000 

\ 3.0 

36754.68 

2.9554 

36744. 4S 

2.9534 

2.9549 

36744.47 l 

2029 

2000 

1999 

2000 

T 3.5 

31824.36 

3.4615 

31815.51 

3.4598 

3.4608 

31815.52 ! 

2022 

2000 

2000 

2000 

! 4.0 

28036.89 

3.9662 

28029.09 

3.9646 

3.9659 

28029.09 J 

2017 

2000 

2000 

2000 

i 4.5 

25042.49 

4.4699 

25035.53 

4.4684 

4.4707 

25035.51 i 

2014 

2000 

2000 

2000 

S 5.0 

22619.35 

4.9729 

22613.07 

4.9715 

4.9730 

22613.04 I 

2011 

2000 

2000 

2000 

brj 1 6.0 

18943.39 

S . 9775 

18938.13 

5.9762 

5.9772 

18938.10 ! 

2008 

2000 

2000 

2000 

| S 7.0 

16291.75 

6.9808 

16287.22 

6.9796 

6.9803 

16287.19 ! 

2006 

2000 

2000 

2000 

U1 J 8.0 

14291.46 ! 

1 7.9833 

14287.48 

! 7.9821 ! 

7.9827 

14287.45 i 

2005 

2000 

2000 

2000 

: ?.o 

12729.82 1 

I 8.9852 

12726.28 

8.9841 ! 

8.9846 

12726,25 ! 

2004 

2000 

2000 

2000 

: io.o 

11477.90 ! 

1 9.9868 * 

11474.72 

1 9.9857 j 

I 9.9861 

11474.69 1 

2003 

2000 

2000 

2000 

! 12.0 

9597.42 1 

11.9892 

9594.76 

11.9882 I 

11.9884 

9594,74 ! 

2002 

2000 

2000 

2000 

i 14.0 

8253.89 I 

! 13,9909 

8251, S9 

t 13,9899 ! 

! 13.9901 

8251.58 i 

2002 

2000 

2000 

2000 

: i6.o 

7247.46 1 

15.9921 

7245.45 

15.9912 1 

15.9914 

7245.44 J 

2001 

2000 

2000 

2000 

: i8.o 

6466.62 ! 

1 17.9931 

6464.82 

1 17.9922 

I 17.9924 

6464.81 ! 

2001 

2000 

2000 

2000 

! 20.0 

5843.90 1 

! 19.9940 

5842.28 

! 19.9931 1 

! 19.9932 

5842.27 i 

2001 

2000 

2000 

2000 

! 25.0 

4730.96 

I 24.9954 

4729.65 

! 24.9946 i 

! 24.9947 

4729.64 1 

2001 

2000 

2000 

2000 

: 30.0 

3999.52 { 

I 29.9964 

3998.41 

1 29.99S6 1 

I 29.9957 

3998.40 i 

2001 

2000 

2000 

2000 

! 35 . 0 

3486.97 ! 

1 34.9971 

3486.00 

! 34.9964 1 

I 34.9965 

3486.00 { 

2001 

2000 

2001 

2000 

1 40.0 

3111.67 j 

1 39.9976 

3110.81 

! 39.9970 { 

39.9970 

3110.80 i 

2001 

2000 

2000 

2000 

! 45.0 

2828.84 ! 

! 44.9981 

• 2828.05 

1 44.9975 

! 44.9975 

2323.05 J 

2001 

2000 

2001 

2000 

J 50.0 

2611.17 1 

49.9984 

2610.45 

49.9979 I 

49,9979 

2610.44 ! 

2000 

2000 

2000 

2000 

i 60.0 

2309.88 1 

1 59.9990 

2309,24 

! 59.9985 ! 

! 59.9986 

2309.23 { 

2001 

2000 

2000 

2000 

i 70.0 

2128.89 i 

69.9994 

2128.30 

69.9991 J 

69.9991 

2128.29 ! 

2001 

2000 

2001 

2000 

! 80.0 

2031.24 1 

i 79.9997 

2030.68 

t 79.9996 ! 

I 79.9996 

2030.67 J 

2000 

2000 

2000 

2000 

1 90.0 

• 

i 

2000.40 ; 

1 

90.0000 

1 

1999.84 

90.0000 1 

i ! 

90,0000 

1999.84 : 

a 

2000 

2000 

2000 

i 

2000 

i 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL COLD-MOIST MORNING SEA LEVEL INDEX OF REFRACTION! .0003226 SCALE HEIGHT! 6779.43 METERS 


1 INPUT 

DATA ! 

GMD GRAD 

REF SOL 1 

CYBER 

SPHERICAL SLAB 1 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD ! 

l E MEAS 
1 DEG 

R hEAS 1 

METER 

E COR 
DEG 

R COR 1 

METER 

E COR 
DEG 

E COR 
DEG 

R COR i 

METER i 

NO CORR 
METER 

GMD H 
METER 


: 

CYB H 
METER 

l 

i 

i 

VAR H 1 

METER i 

\ 0.0 

295927.73 ! 

- .3601 

• 

295848.90 

- . 4325 

-.3075 

i 

295856.37 ! 

6861 

5000 


! 

4630 

« 

* 

\ 

5272 

1 .5 

227740.02 

.2361 

227682.76 

.2124 

.2318 

227681.34 ! 

6050 

5000 


. 

i 

4908 

* 

i 

4983 

: i.o 

178953.96 

.7983 

178910.35 

.7903 

.8006 

178910.18 i 

5631 

sooo 


i 

4977 

• 

i 

5007 

\ 1.5 

144225.35 

1.3401 • 

144190.81 

* 1.3377 

1.3448 

144191.32 i 

5404 

5000 


l 

4996 

S 

5012 

\ 2.0 

119233.42 

1.8691 

119205.16 

1,8690 

1.8729 

119205.87 i 

5273 

5000 


! 

5001 

i 

• 

5008 

l 2.5 

100857.30* 

■ 2.3900 ' 

100833.55 

2.3907 

2.3919 

100834.24 S 

5195 

5000 


i 

5003 

« 

i 

5003 

! 3.0 

86997.78 ! 

2.9056 

86977.37 ! 

2.9066 

2.9059 

86977.96 i 

5144 

SOOO 



5003 

J 

5001 

! 3.5 

7 6278.63 

3.417S 

76260.78 

3.4187 

3.4174 

76261.25 i 

5111 

5000 


1 

1 

5003 

i 

« 

5000 

! 4.0 

67794.95 i 

3.9269 

67779.11 1 

3.9281 

3.9277 

67779.48 l 

5087 

5000 



5003 

1 

5001 

; 4.5 ■ 

60942.31 

4.4345 

60928.10 

4.4356 

4.4375 

60928.39 i 

5071 

5000 


i 

5002 

• 

i 

5003 

i 5.0 

55307.63 S 

4.9407 

55294.74 1 

4.9417 

4.9436 

55294.96 1 

5058 

5000 



5002 

i 

5003 

: 6.o 

46618.97 

5.9503 

46608.12 

5.9511 

5.9518 

46608.26 I 

5041 

5000 


i 

5002 

! 

5001 

! 7.0 

40255.45 l 

6.9573 

40246.09 

6.9579 

6.9580 

40246.19 1 

5031 

5000 



5002 

« 

5001 

1 8.0 

35407.33 

7.9626 

35399*. 10 

7.9631 

7.9629 

35399.19 i 

5024 

5000 


i 

5001 

• 

■ 

5000 

! 9.0 

31597.53 J 

8.9668 

31590.19 ! 

8.9672 

8.9668 

31590.27 ! 

5019 

5000 



5001 

i 

i 

5000 

5 10,0 

28528.57 I 

9.9702 

* 28521.95 1 

1 9.9705 

I' 9.9701 

28522.03 1 

* 5016 

5000 


• 

5001 

* 

i 

5000 

1 12.0 

23896.84 5 

11.9753 

23891,29 i 

1 11.9755 

! 11.97S0 

23891.37 ! 

.5011 

5000 


i 

5001 

• 

i 

5000 

1 14.0 

20573.78 1 

13.9790 

20569.01 

! 13.9791 

! 13.9786 

20569.08 l 

5008 

5000 


« 

5001 

• 

i 

5000 

i 16.0 

18078.10 i 

15.9818 

18073,91 i 

I 15.9818 

! 15.9813 

18073.97 i 

5007 

5000 


■ 

5001 

i 

i 

5000 

} 18.0 

16138.15 1 

17.9840 

16134.41 j 

1 17.9839 

! 17. 9835 

16134.46 i 

5005 

5000 


i 

5001 

. 

• 

5000 

; 20.0 

14589.26 1 

‘ 19.9858 

14585.87 ! 

19.9856 

19.9852 

14585.92 ! 

5005 

5000 


« 

5001 

i 

i 

5000 

J 25.0 

11817.30 1 

24.9890 

* 11814.56 1 

! 24.9888 

I 24.9884 

' 11814.60 1 

5003 

5000 



5001 

i 

• 

5000 

1 30.0 

9993,15 I 

29.9912 

9990.84 1 

29.9909 

29.9907 

9990.87 I 

5002 

SOOO 


» 

5001 

i 

500 0 

! 35 . 0 

8713.76 1 

34.9928' 

8711.74 ! 

! '34.9925 

! 34.9923 

8711.77 1 

5002 

5000 


i 

• 

5001 

■ 

* 5000 

! 40.0 

7777.00 i 

39.9941 

7775.20 S 

39.9938 

39.9936 

7775.23 ! 

5002 

5000 


• 

5001 

* 

50 0 0 

! 45.0 

7070.50 ! 

44.9950 

7068.86 ! 

! 44.9948 

1 44.9946 

7068.89 J 

5002 

5000 


i 

5001 

i 

50 0 0 

J 50.0 

6527,07 ! 

49.9959 

6525.56 ! 

49.9956 

49.9955 

6525.58 ! 

5001 

5000 


.* 

5001 

■ 

50 0 0 

; 60.0 

5774.19 ! 

59.9972 

5772.85 I 

t 59.9970 

! 59.9969 

5772.87 1 

5001 

5000 


i 

5001 

i 

50 00 

: 70.0 

5321.81 J 

69.9983 

5320.57 1 

69.9981 

1 69.9980 

5320.59 ! 

5001 

5000 



5001 

i 

50 0 0 

: eo.o 

5078.21 1 

79.9992 

5077.03 

79.9991 

1 79.9990 

5077.05 1 

5001 

5000 


j 

5001 

i 

50 0 0 

J 90.0 

J 

5001.12 J 

90.0000 

4999.96 J 

90.0000 ] 
l 

1 90.0000 
• 

1 

4999.98 J 
* 
» 

5001 

5000 


i 

• 

5001 

l 

• 

« 

5000 

1 


It**** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 




***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 












COMPARISON OF GRADIENT (GMD > , UHITE SANDS (CYBER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL COLD-MOIST MORNING SEA LEVEL INDEX OF REFRACTION i .0003226 SCALE HEIGHT! 6779.43 METERS 


INPUT 

DATA i 

GMD GRAD 

REF SOL 1 

CYBER 1 

SPHERICAL SLAB i 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E KEAS 

R MEAS 1 

E COR 

R COR 1 

E COR 

E COR 

R COR 1 

NO CORR 



GMD H 



CYB H 


VAR H 

DEG 

METER 

DEG 

METER 

DEG 

DEG 

METER 1 

METER 



METER 



METER 


METER 

0.0 

572003.46 ! 

-.5619 

571897.98 ! 

-.8359 

-.4182 

i 

571924.48 i 

25598 



20000 



17280 


21431 

.5 

499816.55 

.0521 

499734.69 

“.0431 

.1211 

499741.70 1 

23901 



20000 



19179 


20600 

1.0 

440181.74 

.6310 

440115.51 

.5999 

.6795 

440120.97 1 

22831 



20000 



19767 


20372 

1 . S 

390273.53 

1.1886 

390218.36 

1.1846 

1.2252 

390223.10 i 

22118 



20000 



19977 


20249 

2.0 

348256.22 

1.7323 

348209.23 

i .7406 

i .7598 

348213.39 *, 

21625 



20000 



20054 


20167 

2.5 

312744.26 

2.2664 

312703.52 

2.2803 

2,2865 

312707.10 i 

21274 


! 

20000 



20080 


20110 

3.0 

282613.86 

2.7935 

282578.02 

2.8099 

2.8083 

282581.04 1 

21017 



20000 



20084 


20073 

3.5 

256934.80 

3.3154 

256902.88 

3.3327 

3.3275 

256905.42 ! 

20827 


i 

20000 



20080 


20054 

4.0 

234936.50 

3.8336 

234907.79 

3.8508 

3.8459 

234909.93 ! 

20682 



20000 



20073 


20050 

4.5 

215985.00 

4.3487 

215958,94 

4,3655 

4.3644 

215960.76 1 

20570 



20000 



20066 


20059 

5.0 

199561.46 

4.8615 

199537,63 

4.8777 

4.8778 

199539.21 1 

20482 



20000 



20059 


20057 

6,0 

172686.28 

5.8818 

172665.96 

S . 8966 

5.8937 

172667.20 ! 

20356 



20000 



20047 


20036 

7.0 

151786.74 

6.8972 

151769.06 1 

6.9106 

6.9064 

151770.13 ! 

20272 



20000 



20038 


20024 

8.0 

135180.22 

• 7.9092 

135164.58 I 

7.9214 

7.9166 

135165.55 S 

20214 


1 

20000 



20031 


20017 

9.0 

121728.50 

8.9188 

121714.49 i 

8.9299 

8,9250 

121715.40 1 

20172 



20000 



20026 


20013 

1 0 ; 0 * 

110645.89 

9.9267 

110633.21 I 

I 9.9368 

9.9320 

• 110634.06 { 

20141 


i 

i 

20000 



20022 


20010 

12.0 

93528.87 

11.9388 

93518.21 1 

11.9473 

11.9428 

93518.96 i 

20100 


I 

20000 

! 


20016 


20007 

14.0 

80982.61 

13.9476 

80973,40 

13.9550 

! 13.9508 

80974.08 J 

20074 


1 

20000 



20012 


20 0 OS 

16,0 

71428.86 

15.9543 

71420.76 

15.9608 

15.9570 

71421.36 ! 

20057 



20000 

i 


20010 


20003 

18. 0 

63932.05 

17,9596 

63924.81 

17.9653 

! 17.9619 

63925.36 J 

20045 



20000 



20008 


20003 

20 . 0 

57906.08 

19.9639 

57899.53 

19.9690 

19.9659 

57900.04 i 

20036 


i 

20000 

i 


20007 


20002 

£5,0- 

47042.71 

24.9718 

47037.40 

24,9758 

! 24.9733 

47037.81 i 

‘ 20023 


1 

20000 



20005 


20001 

30 . 0 

39845.70 

29.9773 

39841.21 

29.9804 

29.9784 

39841.56 I 

20016 



20000 



20 0 04 


20001 

3S . 0 

34778.47 

34.9813 

34774. S5 

34.9838 

I 34,9822 

34774.86 1 

20012 


1 

1 

20000 



20 0 04 


20 0 01 

40 . 0 

31059.20 

39.9844 

31055.70 

39.9865 

39.9851 

31055.98 { 

20009 


I 

1 

20000 



20 0 03 


20 000 

45 . 0 

28249.78 

44.9869 

28246.60 

44.9887 

I 44.9875 

28246.85 : 

20007 


1 

20000 



20 0 03 


20000 

50 . 0 

26086.57 

49,9890 

26083.63 

49 .9905 

49,9895 

26083.87 i 

20005 



200 00 



20 0 03 


20000 

60 . 0 

23086.32 

59.9925 

23083.72 

59.9935 

! 59.9928 

23083.93 ! 

20004 


l 

20000 



20 0 02 


2 0 0 0 0 

70 . 0 

21282.18 

69.9953 

21279.78 1 

69.9959 

69.9954 

21279.97 J 

20003 


s 

20 000 



20002 


on nnn 

80 . 0 

20309.96 

79.9977 

20307.68 1 

79.9980 

! 79.9978 

20307.86 J 

20002 



20000 



20 0 02 


uUUUU 
20 0 0 0 

90 . 0 

20002.25 

1 

90.0000 

19999.99 1 

90 .0000 

: 90.0000 

i 

20000.17 S 

20002 


i 

• 

20000 



20002 


20000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT <GMD) , WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL COLD-HOIST MORNING SEA LEVEL INDEX OF REFRACTION! .0003226 SCALE HEIGHT! 6779.43 METERS 


INPUT 

DATA 

i GMD GRAD 

REF SOL 

CYDER 

SPHERICAL SLAB l 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E MEAS 

R MEAS 

I E COR 

R COR 

E COR 

E COR 

R COR 1 

‘ NO CORR 

GMD H 

CYB H. 

VAR H 

deg 

METER* 

•1 DEG 

METER 

\ DEG 

S DEG 

* METER i 

METER 

i METER 

METER 

METER 

0.0 

877535.44 

i 

i -.6634 

877425, 9S 

1 

-1.2825 

S 

-.5218 

l 

877456.04 \ 

60085 

50000 

40598 

52153 

* • . 5 * 

804084.51 

! -.0342 

803999,72 

-.2557 

.OiiS 

804005.75 I 

57443 

50000 

46923 

50635 

i.O 

740766.02 

1 .5558 

740697,45 

' .4648 

. S903 

740702.34 \ 

55702 

! 50000 

48838 

50440 

1.5 

685028.67 

! 1.1221 

684971.52 

1.0861 

1.1506 

6S497S.VS ! 

54486 

SOOOO 

49579 

50337 

2.0 

635431.64 

! 1.6726 

635382.90 

i . 6630 

1 . 6957 

635386.87 \ 

53604 

S SOOOO 

49898 

50253 

£.5 

591032.53 

i 2.2123 

590990 .22 

2.2160 

2.2301 

590993.6 7 i 

52945 

50000 

50041 

50181 

3,0 

551143.01 

1 2.7442 

551105,74 

2.7548 

2.7579 

551108.66 ! 

52441 

50000 

50105 

50130 

3.5 

515215.56 

! 3.2702 

515182.32 

3.2845 

3 .2822 

515184.76 ! 

52050 

50000 

50131 

50106 

4.0 

482708.94 

! 3.7918 

482759.00 

3.8079 

3.8054 

482761.05 i 

51740 

50000 

50137 

50113 

4.5 

453474.37 

l 4.3099 

4S3447 . 16 

4.3267 

4.3286 

453448.91 ! 

51492 

50000 

50136 

50146 

5.0 

426914.60 

• 4.8254 

426889.70 

4.8423 

4.8452 

426891.20* ! 

51289 

50000 

50128 

50146 

6.0 

380902.35 

i 5.8501 

380881.08 

5.8665 

5.8653 

380882.26 1 

50985 

50000 

50110 

50098 

7.0 

342719.00 

! 6.0691 

342700.47 

6.8845 

6.8813 

342701.50 i 

50773 

50000 

50092 

50070 

8.0 

310763,33 

i 7.8840 

310746.92 

7.8982 

7.8942 

310747.87 i 

50620 ! 

! 50000 

50077 

50053 

' 9.0 

283782.61 

l 8.8960 

283767.90 

8.9092 

8.9048 

283768.80 ! 

50507 

i 50000 

50066 

50042 

10.0 

260806.13 

i 9.9059 

260792.81 

9.9181 

9.9136 

260793.66 i 

50421 j 

t 50000 

50057 

50034 

12.0 

223973.38 

! 11.9211 

223962.16 

11.9316 

11.9273 

223962.93 \ 

50301 

: 50000 

50041 

50023 

14.0 

195975.35 

! 13.9323 

195965.67 

13.9415 

13.9375 

195966.36 1 

50224 ! 

i 50000 

50031 

50015 

16.0 

174116.06 

l IS. 9409 

174107.53 

15.9490 

15.9453 

174108,16 i 

50172 

: 50000 

50024 

soon 

18.0 

156655.43 

i 17.9477 

156647.81 

17.9549 

17.9516 

156648,37 l 

50136 ! 

i 50000 

50020 

50009 

20.0 

142436.94 

I *19.9532 

142430.04 

19.9597 

19.9566* 

142430.56 I 

50110 ! 

1 50000 

50016 

50007 

25.0 

116424.46 

J 24.9634 

116418.87 

24.9685 

24.9660 

116419.30 ! 

50069 : 

50000 

50012 

50005 ! 

30 . 0 

98943.91 

1 29.9704 

98944.17 

* 29.9745 

29.9725 

* 98944.54 ! 

50046 

t 50000 

50008 

50003 

35.0 

86545.64 

! 34.9756 

86541.51 1 

34.9790 

34.9773 

86541.83 ! 

50031 ! 

50000 

50006 

5oooi : 

40.0 

77395.58 

! 39.9796 

77391.89 1 

39.9824 

39.9810 

77392.19 i 

50022 ! 

! 50000 

50004 

50001 

45.0 

70460.57 

1 44.9829 

70457.22 { 

44.9853 

44.9841 

70457.48 l 

50016 ! 

1 50000 

50004 

50000 

50 . 0 

65108.19 

{ 49.9856 

*65105.09 i 

49.9876 

49.9B66 

65105.34 ; 

50012 

; soooo 

50003 

50000 

60.0 

57668.24 

1 59.9901 

57665.51 ! 

59.9915 

59.9908 

57665.72 ! 

50007 ! 

50000 

50003 

50000 

70.0 

53185.38 

! 69.9938 

53183.36 1 

1 69.9946 

69.9942 

53183.56 \ 

* 50004 

! 50000 

50002 

50000 

80.0 

50768.01 

1 79.9970 

50765.61 ! 

79.9974 

79.9972 

50765.80 1 

50003 ! 

1 50000 

50002 

50000 

90.0 

50002.38 

: 90.0000 
< 

• 

50000.00 i 
■ 
i 

! 90.0000 

\ 

90.0000 

50000.19 J 
• 
1 

50002 

! 50000 

50002 

i I 

50000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 
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COMPARISON OF GRADIENT (GMD), WHITE SANDS < CYDER ) AND SPHERICAL SLAB ( VAR I AN ) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL COLD-MOIST MORNING SEA LEVEL INDEX OF REFRACTIONi .0003226 SCALE HEIGHT! 6779.43 METERS 


INPUT 

DATA 

GMD GRAD REF SOL 

CYBER 

SPHERICAL SLAB \ 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E MEAS 
DEG 

R f MEAS 
METER 

E COR 
DEG 

R COR 
METER 1 

E COR 
DEG : 

E COR 
DEG 

R COR \ 

METER i 

NO CORR 
METER 


GMD H 
METER 

1 

1 

CYB H 
METER 

i 

i 

i 

VAR H ! 

METER J 

0.0 

1216581,21 

-.7170 

1216470.33 

« 

-1.7780 i 

-.5806 

1 

ft 

1216501.74 .* 

114990 


100000 


77806 


102854 

.5 

1142531,06 

-.0769 

1142445.52 

-.4336 ! 

-.0513 

1142451.95 S 

111330 


100000 


93007 


100503 

• i .0 

1077440.99 

.5207 

1077371.97 1 

.3669 ! 

.5420 

1077376.20 * 

108B79 


100000 


97164 


100395 

i .5 

1018814.72 

1.0924 

1018757.28 

• 1.0212 

1.1126 

1018761.34 1 

107140 


100000 


98764 


100354 

2.0 

965291.67 

1.6471 

965242.75 

1.6152 I 

1.6647 

965246.50 1 

105855 


100000 


99478 


100291 

2,5 

916066, '17 

■ 2.1900 

916023.74 

2.1785 

2.2041 

916027.05 J 

104881 


100000 


99826 


100222 

3.0 

870554.63 

2.7244 

870517.27 

2 .7240 i 

2.7356 

870520.10 ! 

104122 


100000 


99999 


100166 

3.5 

828360.50 

3.2524 

828327.20 

3.2583 

3.2628 

828329.58 ! 

103522 


100000 


100088 


100146 

4.0 

789167,33 

3.7757 

789137.34 

3.78S1 S 

3.7885 

789139.35 ! 

103039 


100000 


100132 


100173 

4.5 

752706.92 

4.2952 

752679.67 

4.3066 

4.3140 

752681.39 1 

102645 

I 

100000 


100152 


100243 

5.0 

718748.24 

4.8118 

718723.30 

4.8242 ! 

4.8322 

718724.78 I 

102320 


100000 


100157 


100251 

6.0 

657566.39 

5.8384 

657545.10 

5.0515 

5.8543 

657546.26 ! 

101818 


100000 


100149 


100177 

7.0 

604261 .28 

6.8588 

604242.73 

6.8715 l 

6.8717 

604243.74 1 

101458 


100000 


100134 


100132 

8.0 

557678.14 

7.8748 

557661.72 

7. 8869 

7.8858 

557662.65 i 

101192 


100000 


100118 


100104 

9.0 

516813.62 

8.8877 

516798.91 

8.8991 1 

8.8973 

516799.79 ! 

100986 


100000 


100101 


100082 

l 10.0 

480861,80 

9.8984 

480848.47 

• 9.9090 1 

9.9068 

480849.31 1 

10 0830 


100000 


100090 


100069 

i 12.0 

420899.89 

11.9148 

420888.67 J 

11.9240 : 

11.9216 

420889.43 I 

100604 

l 

100000 


100067 


100047 

1 14.0 ' 

373347.66 

t 13.9268 

373337.97 ! 

! 13.9350 1 

13.9326 

373338.65 ! 

100457 


100000 


100052 


100035 

! 16.0 

335024.92 

15.9361 

335016.39 J 

15.9433 1 

15.9411 

335017.01 ! 

100355 


100000 


100041 


100026 

I 18.0 

303680 . 02 

17.9434 

■303672.40 

17.9499 ! 

17.9478 

303672,96 t 

100283 


100000 


100035 


100022 

1 20.0 

277677.73 

19.9494 

277670.83 1 

19.9552 1 

19.9533 

277671.35 \ 

100228 


100000 


100027 


100016 

\ 25.0 

229084.02 

24.9603 

229078.43 

24.9649 ! 

24.9634 

229078.85 i 

100143 


100000 • 


100018 


100010 

i 30.0 

195752.11 

29.9679 

19S747 , 37 

29.9717 1 

29.9703 

195747.74 ♦ 

100095 


100000 


100012 


100006 

i 35,0 

171793.66 

34.9735 

171789.53 

34.9766 ! 

34.9755 

171789.85 S 

100065 


100000 


100009 


100004 

! 40.0 

153973.74 

39.9779 

153970.05 

39.9805 1 

39.9795 

153970.34 { 

100046 


100000 


100007 


100003 

! 45.0 

140392.51 

44.9814 

140389,16 

44.9836 

44.9828 

140389.42 1 

100033 


100000 


100005 


100002 

i 50.0 

129870 .85 

49.9844 

129867.75 

49.9862 1 

49.9856 

129867.99 \ 

100025 


100000 


100005 


100002 

! 60.0 

115189.05 

59.9893- 

115186.31 

59.9905 

59.9901 

115186.53 1 

100013 


100000 


100003 


100001 

! 70.0 

106316.02 

69.9932 

106313.50 1 

69.9940 S 

69.9937 

106313.70 \ 

100006 


100000 


100003 


100000 

: 80.0 

101521,67 

79.9967 

101519.26 

79.9971 

79.9970 

101519.45 i 

100003 


100000 


100002 


100000 

! 90.0 

1 

100002.38 

90.0000 

100000.00 1 

90.0000 : 

90.0000 

100000.19 i 
\ 

100002 

1 

1 

100000 


100002 


100000 

i 


It**** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



APPENDIX G 


TABULATED REFRACTION COMPARISON DATA 
FOR EDWARDS TYPICAL WARM MOIST MORNING 
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TABLE G-I 


WEATHER DATA FOR TYPICAL WARM MOIST MORNING 


ALTITUDE 

(FT) 

TEMPERATURE 
(DEG C) 

DEW POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 1 0E6 

2316 

17.00001 

15.84997 

929.00000 

329.30969 

3000 

29.00000 

-.15002 

903.26001 

257.02301 

4000 

27.00000 

-16.15002 

891.03003 

237.67468 

sooo 

24.50000 

-17.15002 

857.80005 

230.47379 

6000 

22.00000 

-18.15002 

819.53003 

221.86047 

7000 

19.50000 

-19.15002 

794.59998 

216.67038 

8000 

17.00000 

-20.15002 

768.16003 

211.01862 

9000 

14.50000 

-21.15002 

739.39001 

204.66995 

i 00 00 

12.00000 

-22.15002 

713.06006 

198.90247 

1 i 0 0 0 

9.50000 

-23.15002 

688.07996 

193.43057 

12000 

7.00000 

-25.15002 

663.22998 

187.56149 

13000 

4.50000 

-27.15002 

639.75000 

182.07138 

14000 

2.00001 

-29.15002 

617.23010 

176.84204 

15000 

-.50000 

-31.15002 

594.35999 

171.49615 

16000 

-3.00000 

-33.15002 

572.38000 

166.37622 

17000 

-5.50000 

-36.15002 

551.87000 

161.49234 

18000 

-8.00000 

-39.15002 

531.03003 

156.53363 

19000 

-10.50000 

-42,15002 

510.69000 

151.71918 

20000 

-13.00000 

-45.15002 

491.85999 

147.33475 

21000 

-15.70001 

-48.15002 

473.03998 

143.03632 

22000 

-18.40002 

-51 . 15002 

454.40997 

138.74869 

23000 

-21.30002 

-54.15002 

437.02002 

134.89230 

24000 

-24.00003 

-57.15002 

420 . 14001 

131 . 02570 

25000 

-26.70001 

-60.15002 

403.20001 

127.07535 

26000 

-29.50003 

-64.15002 

387.12000 

123.36671 

27000 

-32.30000 

-68.14999 

372.11011 

119.93466 

28000 

-35.10002 

-72.15001 

356.70007 

116.30334 

29000 

-37.90002 

-76.15002 

341.96997 

112.81746 

30000 

-40.70001 

-80.15002 

328.31000 

109.60959 

31000 

-40.67321 

-78.92619 

328.18799 

109.49875 

32000 

-43.26289 

-78.77321 

315.28833 

106.24097 

33000 

-47.19596 

-79.23215 

296.06165 

101.44191 

34000 

-51.19937 

-79.84407 

276.95868 

96,70723 

35000 

-54.00003 

-80.15002 

264.42999 

93.64259 

36000 

-56.30975 

-80,15002 

255.89999 

91.58688 

37000 

-59,35925 

-80,15002 

245.22552 

88.96097 

38000 

-62.65390 

-80 . 15002 

233.50607 

85.99347 

39000 

-65.69904 

-80.15002 

221.84106 

82.91298 

40000 

-68.00003 

-80.15002 

211.32999 

79.94809 

41000 

-69.18581 

-80.15002 

201.81140 

76.86771 

42000 

-69.44051 

-80 . 15002 

192.46701 

73.42751 

43000 

-69.10233 

-80.15002 

183.41089 

69.83629 

44000 

-68.50945 

-80 . 15002 

174.75720 

66.30289 

45000 

-68,00003 

-80.15002 

166.62000 

63.03609 

46000 

-67.35915 

-80 . 15002 

158.83591 

59.92341 

47000 

-66.26350 

-80,15002 

151.21689 

56.76656 

48000 

-64.88873 

-80 . 15002 

143.84003 

53.64563 

49000 

-63.41046 

-80.15002 

136.78241 

50.64070 

50000 

-62.00433 

-80 . 15002 

130 . 12112 

47.83187 
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TABLE G-l — Continued 


WEATHER DATA FOR TYPICAL WARM MOIST MORNING 


ALTITUDE 
(FT) • 

TEMPERATURE 
(DEG C) 

DEU POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 1 0E6 

51000 

-60.S689S 

-80.15002 

123.71893 

45.17155 

520 00 

-58.98966 

-80.15002 

117.44495 

42.58508 

53000 

-57.42764 

-80 . 15002 

111.38992 

40.11436 

540 00 

-56.04404 

-80.15002 

105.64471 

37.80131 

55000 

-55.00003 

-80.15002 

100.30000 

35.68780 

56000 

-54.47746 

-80.15002 

95.31410 

33.79884 

57000 

-54,41688 

-80.15002 

90.58319 

32.10184 

58000 

-54.61759 

-80.15002 

86.10529 

30.55256 

59000 

-54.87888 

-80.15002 

81.87820 

29.10661 

60000 

-55.00003 

-80.15002 

77.89999 

27.71970 

61000 

-54.98045 

-80.15002 

74.13089 

26.37768 

62000 

-54.96028 

-80.15002 

70.53699 

25.09793 

63000 

-54.94990 

-80.15002 

67 . 12457 

23.88339 

64000 * 

-54.95969 

-80.15002 

63.90015 

22.73705 

65000 

-55.00003 

-80.15002 

60.87000 

21.66182 

66000 

-55.07311 

-80.15002 

SS. 00195 

20.64641 

67000 

-55.14839 

-80.15002 

55.26201 

19.67670 

68000 

-55.18713 

-80.15002 

52.65412 

18.75158 

69000 

-55.15058 

-80.15002 

50.18214 

17.86984 

70000 

-55.00003 

-80.15002 

47.85000 

17.03036 

71000 

-54.62611 

-80.15002 

45.62428 

16.21456 

72000 

-54.02283 

-80.15002 

43.47842 

15.41321 

73000 

-53.30652 

-80.15002 

41.42638 

14.63907 

74000 

-52.59348 

-80.15002 

39.48225 

13.90495 

75000 

-52.00003 

-80.15002 

37.66000 

13.22361 

76000 

-51.53118 

-80.15002 

35.93859 

12.59022 

77000 

-51.10735 

-80.15002 

34.29008 

11.99045 

78000 

-50.71793 

-80.15002 

32,71828 

11.42289 

79000 

-50.35236 

-80.15002 

31.22698 

10.88610 

80000 

-50 . 00003 

-80.15002 

29.82000 

10.37866 

81000 

-49.62761 

-80.15002 

28.47564 

9.89296 

82000 

-49.20585 

-80.15002 

27,17604 

9.42399 

83000 

-48.73029 

-80.15002 

25.93060 

8.97419 

84000 

-48.19650 

-80.15002 

24.74878 

8.54601 

85000 

-47,60004 

-80.15002 

23.64000 

8.14192 

86000 

-46.86880 

-80.15002 

22.58916 

7.75574 

87000 

-46.01745 

-80.15002 

21.57758 

7.38245 

88000 

-45.17173 

-80.15002 

20.60943 

7.02622 

89000 

-44.45734 

-80.15002 

19.68884 

6.69122 

90000 

-44.00003 

-80.15002 

18.82000 

6.38161 

91000 

-43.87155 

-80.15002 

18.02932 

6.10960 

92000 

-43.63983 

-80.15002 

17.15881 

5.81004 

93000 

-42.94996 

-80.15002 

16.22529 

5.47901 

94000 

-42.11295 

-80.15002 

15.44332 

5.19646 

95000 

— 4i .53740 

-80 . 15002 

14.95721 

5.01962 

96000 

-41 . 17400 

-80.15002 

14.63868 

4.90530 

97000 

-40.48238 

-80.15002 

14.12607 

4.72159 

98000 

-39.56448 

-80.15002 

13.48358 

4.49147 

99000 

-38.53429 

-80.15002 

12.78299 

4.24069 

100000 

-37.50578 

-80.15002 

12.09601 

3.99496 
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COMPARISON OF GRADIENT <GMD>, WHITE SANDS (CYDER > AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM-MOIST MORNING SEA LEVEL INDEX OF REFRACTION i .0003661 SCALE HEIGHT i 5977.06 METERS 



INPUT 

DATA 

1 GMD GRAD 

REF SOL 2 

CYDER 2 

SPHERICAL SLAB ! 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

1 — 
1 

E MEAS 

R MEAS 

1 E COR 

R COR 1 

E COR 2 

E COR 

R COR 1 

NO CORR 1 

GMD H 

l CYB H 


VAR H 

j 

DEG 

METER 

1 DEG 

METER 2 

DEG 

DEG 

METER 2 

METER 

METER 

1 METER 


METER 

\ 

» 

0.0 

143171.41 

1 

J -.2425 

i 

143122.44 1 

******* ! 

-.2467 

1 

143115.05 1 

1607 

1000 

1 

1 ^ ^ ^ ^ ti* d/ i 

ft ^ ^ ^ ^ ( 


989 

! 

.5 

79089.58 

! .3696 

79063.52 2 

******* 

.3638 

79060.07 S 

1180 

1000 

2 ******* 2 

992 

1 

i.O 

50129.67 

S .9184 

50113.34 1 

******* 

.9180 

50112.09 1 

1072 

1000 

2 ******* 2 

1000 

ft 

i .5 

35764.69 

1 1,4422 

35753.09 1 

******* 

1.4426 

35752.49 ! 

1036 

1000 

1 

| ^ ^ ^ ^ 'r 'n 


1000 

i 

« 

2.0 

27567.91 

S 1.9558 

27558.98 2 

******* 

1.9556 

27558.59 2 

1022 

1000 

{ ******* 


1000 

I 

2.5 

22355.21 

2 2.4644 

22347.98 2 

******* 

2.4636 

22347.69 2 

1014 

1000 

2 ******* 


1000 

i 

3.0 

18772.98 

J 2.9703 

18766.91 I 

******* 

2.9690 

18766.67 S 

1010 

1000 

i kkkkkkk 

1 T ^ T ^ ^ 


1000 

; 

3.5 

16168.62 

! 3.4745 

16163.39 ! 

******* 

3.4732 

16163.18 S 

1007 

1000 

2 ******* 


1000 

1 

4.0 

14193.46 

1 3.9778 

14188.87 i 

******* 

3.9768 

14188.68 J 

1006 

1000 

2 ******* 


1000 

1 

4.5 

12645.80 

2 * 4.4803 

12641.71 i 

******* 

4.4802 

12641.54 { 

1005 

1000 

J ******* 


1000 


5.0 

11401.27 

I 4.9824 

11397.59 1 

******* 

4.9815 

11397.43 ! 

1004 

1000 

I ******* 


1000 


6.0 

9525.53 

1 5.9855 

9522.45 1 

******* 

5.9845 

9522.31 i 

1003 

1000 

2 ******* 


1000 


7.0 

8180.04 

1 6.9877 

8177.40 1 

******* 

6.9866 

8177.28 { 

1002 

1000 

} ******* 


1000 


8.0 

7168. S5 

{ 7.9894 

7166.23 1 

******* | 

7.9883 

7166.13 ! 

1002 

1000 

J ******* 


1000 


9.0 

6381 . 06 

2 8.9907 

6379.00 ! 

******* ! 

8.9896 

6378.90 2 

1001 


2 ******* 


1000 

J 

10.0 

5750.74 

I 9.9917 

5748.88 1 

******* : 

9.9906 

5748.80 ! 

1001 


} ******* 


1000 


12.0 

4805.58 

! 11.9933 

4804.03 t 

******* ; 

11.9922 

4803.96 S 

1001 

1000 

2 ******* 


1000 


14.0 

4131.28 

1 13.9944 

4129.95 1 

******* 

13.9934 

4129.89 2 

1001 

1000 

ft ^|c )|C 


1000 


16.0 

3626.63 

! 15.9953 

3625.46 ! 

******* J 

15.9942 

3625.41 • 

1001 

1000 

1 ******* 


1000 


18.0 

3235.27 

2 17.9959 

3234.22 2 

******* : 

17.9949 

3234.18 2 

1000 

1000 

i ^ ^ 5|C 


1000 


20.0 

2923. 4S 

1 19.9964 

2922.50 S 

******* j 

19.9954 

2922.46 ! 

1000 

1000 

2 ******* 


1000 


25.0 

2366.24 

J 24.9974 

2365.48 1 

******* ; 

24.9964 

2365.44 i 

1000 


2 ******* 


1000 


30.0 

2000.11 

1 29.9981 

1999.47 2 

******* | 

29.9971 

1999.44 1 

1000 

1000 

J ******* 


i 00 0 


35 .0 

1743.69 

{ 34.9985 

1743.13 2 

******* 

34.9976 

1743.10 2 

1000 

1000 

2 ******* 


1000 


40.0 

1556.04 

l 39.9988 

1SS5.53 1 

******* j 

39.9980 

1555.51 2 

1000 

1000 

2 ******* 


1000 


45.0 

1414.57 

! 44.9991 

1414.11 1 

******* 

4*4.9983 

1414.09 2 

1000 

1000 

i 


1000 


50.0 

1305.59 

J 49.9993 

1305.17 ! 

******* | 

49.9986 

1305.15 I 

1000 

1000 

2 ******* 


1000 


60.0 

1154.91 

1 59.9995 

1154.55 I 

******* 

59.9990 

1154.52 1 

1000 

1000 

J ******* 


1000 

2 

70,0 

1064.40 

i 69.9997 

1064.07 1 

******* ! 

69.9994 

1064.04 1 

1000 

1000 

i ^ ^ 


1000 

80.0 

1015.66 

1 79,9999 

1015*34 ! 

******* 

79.9997 

1015.32 2 

1000 

1000 

2 ******* 


1000 

I 

90.0 

1000 .24 

! 90.0000 
I 

999.92 ! 

i 

• 

******* j 

90.0000 

1 

999.90 2 
2 

1000 

1000 

i kkkkkkk 


1000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 





COMPARISON OF GRADIENT (GMD) , WHITE SANDS (CYDER) AND SPHERICAL SLAD (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 75-FM-60 


WEATHER DATA FOR TYPICAL WARM-MOIST MORNING SEA LEVEL INDEX OF REFRACTION! .0003661 SCALE HEIGHT: S977.06 METERS 


1 INPUT 

DATA 

GMD GRAD 

REF SOL 

CYDER 

SPHERICAL SLAB i 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

i E MEAS 

R MEAS 

E COR 

R COR 

E COR 

E COR * 

R COR 1 

NO CORR 

GMD H 

CYB H 

VAR H 

! DEG 

METER 

DEG 

METER 

DEG 

DEG 

METER 1 

METER 

METER 

METER 

METER 

I 

! 0.0 

199864.31 

-.3238 

199801.63 

******* 

-.3321 

199797.50 { 

3131 

2000 

******* 

1971 

J .5 

129740.23 

.3011 

129702.25 

******* 

.2805 

129694.78 i 

£451 

2000 

******* 

1953 

i i.O 

90237.98 

.8653 

90212.29 

******* 

..8585 

90208, 3S ! 

2213 

2000 

******* 

1989 

! 1 . 5 

67319.16 

1.4008 

67300,22 

******* 

1.3974 

67297.80 * 

2117 

2000 

******* 

1996 

! 2,0 

53066.09 

1.9224 

53051.24 

******* 

1.9191 

53049.51 J 

2072 

2000 

******* 

1997 

! 2 . 5 

43563.99 

2.4367 

43551.84 

******* 

2.4328 

43550.46 1 

2049 

2000 

******* 

1997 

i 3.0 

36851 .82 

2,9467 

36841.56 

******* 

2.9424 

36840.40 I 

2035 

2000 

******* 

1997 

! 3 . S 

31887.59 

3.4541 

31878.72 

******* 

3.4500 

31877.70 i 

2026 

2000 

******* 

1998 

: 4.o 

28080 . 06 

3,9597 

28072,26 

******* 

3.9566 

28071.34 1 

£020 

2000 

******* 

1998 

! 4,5 

25073.25 

4.4642 

25066.28 

******* 

4.4627 

25065.45 ! 

2016 

2000 

******* 

1999 

O ' 5.0 

22641 .94 

4.9678 

22635.65 

******* 

4.9656 

22634.89 1 

2013 

2000 

******* 

1999 

i r 6,o 

18956.62 

5.9732 

18951,36 

******* 

5.9710 

18950.70 ! 

2009 

2000 

******* 

1999 

i 7.0 

16300.10 

6.9771 

16295.58 

******* 

6.9749 

16295.01 ! 

2007 

2000 

******* 

1999 

: 8.o 

14297,04 

7.9801 

14293.07 ! 

1 1 
j ^ ^ ^ ^ ^ ^ ^ 1 

7.9780 

14292.58 ! 

2005 ’1 

2000 

^ ^ ^ ^ 

1999 

; ?.o 

12733.73 

8.9824 

12730.19 i 

******* ! 

8,9804 

12729.75 ! 

£004 I 

2000 

******* 

1999 

{ 10.0 

11480,75 

9,9843 

11477.56 1 

J ******* J 

! 9.9823 

11477.17 I 

2004 ! 

2000 

^ ^ ^ ^ ^ 

2000 

i 12.0 

9599.04 

11.9871 

9596.38 J 

******* S 

11.9853 

9596.05 l 

2003 ! 

2000 

******* 

2000 

: i4.o 

82S4.89 

13.9891 

8252.60 ! 

[ ******* 1 

! 13.9874 

8252.32 ! 

2002 ! 

2000 

******* 

2000 

i 16.0 

7248.13 

15.9906 

7246.12 I 

! ******* 1 

IS. 9891 

7245.87 S 

2002 ! 

2000 

******* 

2000 

i 18. 0 

6467.07 

t 17,9918 

6465.28 

| ******* ! 

I 17.9903 

6465,06 i 

2001 

2000 

^ ^ ^ ^ ^ 

2000 

! 20.0 

5844.24 

19.9928 

5842.62 ! 

i ******* : 

19.9914 

5842.42 1 

2001 ! 

2000 

******* 

2000 

! 25.0 

4731 . 13 

! 24.9945 

4729,82 

i ******* i 

S 24.9933 

4729.65 ! 

2001 

2000 

******* 

2000 

! 30.0 

3999.61 

! 29.99S7 

3998.50 I 

i ******* j 

1 29.9945 

3998.36 1 

2001 ! 

2000 

******* 

2000 

! 35.0 

3487.03 

» 34.9965 

3486.06 

i ******* 

1 34.9955 

3485.94 ! 

2001 

2000 

******* 

2000 

! 40.0 

3111.71 

! 39,9972 

3110.84 ! 

' ******* j 

! 39.9962 

3110.74 l 

2001 1 

2000 

^ ^ ^ ^ ^ 

2000 

{ 45.0 

2828,85 

1 44.9977 

2828,07 

! ******* 1 

1 44.9969 

2827.97 J 

2001 

2000 

******* 

2000 

J 50.0 

2611 . 19 

49.9981 

2610.47 ! 

; ******* ) 

1 49.9974 

2610.38 i 

2001 ! 

2000 

******* 

2000 

! 60.0 

2309.88 

1 59.9988 

2309,25 

} ******* 

! 59.9982 

2309.16 1 

2001 

2000 

******* 

2000 

! 70.0 

2128.88 

69.9992 

2128,30 1 

1 ******* ; 

! 69.9989 

2128.22 1 

2001 ! 

2000 

******* 

2000 

S 80.0 

2031.25 

[■ 79,9996 

2030,69 ! 

; ******* i 

1 79.9994 

2030.62 1 

2000 

2000 

******* 

2000 

} 90.0 

l 

2000.41 

90.0000 

i 

1999,86 ! 

! ******* j 

■ i 

! 90.0000 
l 

1999.78 1 
\ 

2000 1 

2000 

******* 

2000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COhR ARISON OF GRADIENT (GMD) , WHITE SANDS (CYDER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM-MOIST MORNING SEA LEVEL INDEX OF REFRACTION! .0003661. SCALE HEIGHT! 5977.06 METERS 
I 1 


INPUT 

DATA 

1 

GMD GRAD 

REF SOL 

• 

« 

CYBER 

t 

SPHERICAL SLAB 2 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E HEAS 

R MEAS 

i 

E COR 

R COR 

2 

E COR 

I 

E COR 

R COR 2 

NO CORR 

GMD H 

l CYB H 2 

VAR H 

1 DEG 

METER 

■ 

i 

DEG 

METER 

\ 

DEG 

1 

' DEG 

METER 2 • 

METER 

2 METER 

J METER I 

METER 

! 0.0 

308341.52 

S 

! 

- . 4548 

308256.34 

• 

• 

• 

i 

******* 

1 

I 

-.4187 

1 

308263.75 2 

7449 

l 

5000 

i : 

i ******* ; 

S194 

: .5 

232911.85 

i 

.1849 

232853.31 

• 

i 

******* 

1 

.1462 

232844.77 2 

6282 

1 5000 

■ ******* ; 

4843 

! 1.0 

181402.44 

• 

i 

.7657 

181358 * 90 

* 

******* 

S 

.7424 

181353.43 ! 

5743 

5000 

• **•****■♦ i 

4926 

] l.S 

145517.41 

t 

1.3166 

145483.30 

• 

» 

******* 

1 

1.3019 

145479.77 2 

5467 

1 5000 

l ******* ! 

4963 

! 2.0 

119973.43 

t 

1.8509 

119945.67 

. 

• 

******* 

1 

1.8388 

119943.14 2 

5313 

5000 

I ******* J 

4974 

\ 2.5 

101310.40 

\ 

2.3752 

101287.14 

2 

******* 

1 

2.3634 

1012B5.il 2 

5222 

2 5000 

2 ******* i 

4979 

! 3.0 

87290.83 

s 

2.8932 

87270.88 

• 

• 

******* 

* 

i 

2.8814 

87269.11 . 

5164 

5000 

i ******* 2 

4982 

i 3.5 

76477.23 

i 

3.4068 ; 

76459.80 

i 

i 

******* 

2 

3.3961 

76458.20 2 

5125 

2 5000 

| ******* J 

4986 

: 4.0 

. 6793A . 84 

i 

3.9176 

67919.40 

* 

i 

******* 

. 

i 

3.9092 

67917.91 ! 

5099 

5000 

j ^ % ijc jjc 2 

4990 

i 4.5 

61044.09 

i 

4.4262 

61030.24 

i 

******* 


4.4216 

61028.86 J 

5080 

2 5000 

2 ******* 2 

4995 

! 5.0 

55383.65 

i 

4.9332 

55371.10 

i 

******* 

i 

• 

4.9292 

55369.80 J 

5065 

5000 

t 1 

1 * ^ n' ^ T * 1 

4996 

\ *6.0 

46664.46 

2 

5.9440 

46653.89 

i 

******* 

s 

5.9395 

46652.76 { 

5046 

! 5000 

2 ******* J 

4996 

S 7.0 

40284.56 

1 

6.9519 

40275.45 

2 

******* 

1 

1 

6.9474 

40274.47 S 

5035 

5000 

J ******* ! 

4997 

1 6.0 

35427.01 

2 

7.9579 

35419.00 

i 

******* 

I 

7.9535 

35418.14 1 

5027 

2 5000 

2 ******* { 

4997 

S 9,0 

31611.36 

2 

8.9627 

31604.22 

\ 

******* 

1 

1 

8.9585 

31603.46 S 

5021 

: 5000 

2 ******* ; 

4998 

2 10.0 

28538.66 

1 

9.9665 

28532.22 

• ■ 
• 

******* 

1 

1 

9.9625 

28531.54 1 

5018 

! 5000 

J ******* I 

4998 

{ 12.0 

23902.61 

! 

11.9722 

23897.22 

• 

******* 

( 

< 

11.9687 

23896.66 2 

5012 

2 5000 

1 ^ • 
1 v T ^ ^ T ^ * 1 

4998 

! 14.0 

20577.36 

• 

i 

13.9764 

20572.72 

* 

• 

******* 

1 

1 

13.9732 

20572.24 1 

500? 

! 5000 

1 * 
1 ^ T ^ T* * ^ T 1 

4999 

1 16.0 

18080.45 

• 

> 

15.9795 

18076.38 

• 

• 

******* 

• 

1 

15.9766 

18075.96 2 

5007 

! 5000 

i k k k k k ■ 

1 ^ T T T | 

4999 

i 18.0 

16139.75 

\ 

17.9820 

16136.12 

• 

« 

******* 

1 

1 

17.9793 

16135,74 ! 

5006 

• 5000 

i * k k k k k k • 

4999 

; 20.0 

14590 . 41 

1 

1 

19.9840 

14587.12 

• 

i 

******* 

l 

19.9815 

14586.78 i 

5005 

! 5000 

i i 

1 ^ ^ a* 4K ^ T 1 

4999 

I 25.0 

11817.83 

i 

24.9876 

11815.16 

2 

******* 

1 

i 

24.9855 

11814.89 5 

5003 

I 5000 

2 ******* J 

4999 

1 30.0 

9993.42 

1 

29.9901 

9991.17 

• 

* 

******* 

t 

1 

29.9883 

9990.94 { 

5003 

2 5000 

2 ******* ; 

5000 

! 35,0 

8713.91 

1 

34.9919 

8711.94 

i 

« 

******* 

1 

1 

34.9904 

8711.74 i 

5002 

! 5000 

2 ******* ; 

5000 

; 40.0 

7777 , 08 

* 

39.9933 

7775.32 

« 

******* 

: 

39.9919 

7775.15 1 

5002 

2 5000 

2 ******* 1 

500 0 

! 45.0 

7070 . 53 

1 

1 

44.9944 

7068.94 

• 

i 

******* 

i 

44.9932 

7068,78 i 

5002 

: 5000 

2 ******* t 

50 0 0 

1 50.0 

6527.07 

2 

49.9954 

6525.61 

i 

******* 

• 

i 

49.9943 

6525.46 J 

5001 

i 5000 

2 ******* ; 

50 0 0 

: 6 o.o 

5774.17 

• 

i 

59.9969 

5772.87 

i 

* 

******* 

i 

59.9961 

5772.73 ! 

5001 

! 5000 

2 ******* { 

50 0 0 

! 70.0 

5321 .78 

i 

• 

69.9981 

5320,58 

• 

******* 

i 

t 

69.9975 

5320,46 ! 

5001 

1 5000 

2 ******* 1 

50 0 0 

t 80.0 

5078. 17 

i 

i 

79.9991 

5077.03 

i 

******* 

! 

79.9983 

5076.91 2 

5001 

2 5000 

2 ******* i 

50 0 0 

1 90.0 

! 

5001 . 08 

i 

i 

• 

90.0000 

4999.96 

i 

i 

******* 

• 

i 

90.0000 

4999.84 1 

5001 

: 5000 

2 ******* 2 

5000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD) , WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM-MOIST MORNING SEA LEVEL INDEX OF REFRACTION) .0003661 SCALE HEIGHT) S977.06 METERS 


INPUT 

DATA 

-GMD GRAD 

REF SOL 

* 

CYBER 

SPHERICAL SLAB f 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E HEAS 

R HEAS 

E COR 

R COR 

i 

E COR 

E COR 

R COR * 

NO CORR 

GMD H 

CYB H 

VAR H 

DEG 

• HETER 

DEG 

METER 

i 

DEG 

DEG 

HETER ! 

METER 

METER 

METER. 

METER 

0.0 

426524.78 

-.5706 

426421.74 

! 

1 

******* 

-.4901 

i 

426442.66 \ 

14246 

10000 

******* 

10600 

. 5 

348571.75 

.0801 

348496.78 

i 

******* 

.0702 

348494.13 \ 

12554 

10000 

******* 

9940 

i.O 

290413,89 

.6707 

290355.34 

\ 

******* 

.6583 

290353.67 1 

11669 

10000 

******* 

9937 

' l.S 

245674.49 

1.2313 

245626.83 

\ 

******* 

1.2214 

245626. 2S \ 

111S3 

10000 

******* 

9957 

2.0 

210777,38 

1.7747 

210737.47 

* 

i 

******* 

1.7651 

210737.41 1 

10830 

10000 

******* 

9965 

2.5 

183263.01 

2.3071 

183228.87 

1 

******* 

2.2965 

183228.93 \ 

10618 

10000 

******* 

9966 

3.0 

161312.45 

2,8320 

161282.73 

i 

******* 

2.8207 

161282.74 i 

10474 

10000 

******* 

9968 

3.5 

143574.42 

3,3517 

143548.17 

* 

******* 

3.3412 

143S48 . 09 ! 

10373 

10000 

******* 

9974 

4.0 

129050.04 

3.8675 

129026.58 

\ 

******* 

3.8599 

129026.40 I 

10299 

10000 

******* 

9983 

4.5 

117004,25 

' 4.3805 

116983.07 

1 

******* 

4.3781 

116982.84 » 

10245 

10000 

******* 

9995 

5.0 

106892.06 

4.8912 

106872.78 

i 

******* 

4.8909 

106872.50 I 

10204 

10000 

******* 

9999 

6.0 

90946.42 

5.9081 

90930.08 

1 

******* 

5.9059 

90929.79 i 

10147 

10000 

******* 

9997 

7.0 

79011.34 

6.9206 

78997.19 

\ 

******* 

6.9177 

78996.93 ! 

10110 

10000 

******* 

9996 

8.0 

69784.53 

I 7.9302 

69772.05 

i 

* 

******* | 

7 . 9270 

69771.85 1 

10086 

10000 

******* 

9996 

9.0 

62459.34 

8.9378 

62448.19 

i 

******* \ 

8.9346 

62448.04 i 

10069 

10000 ! 

******* 

9996 

10.0 

* 56514.71 

1* 9.9440 

56S04.63 

i 

******* j 

1 9.9408 

56504.52 1 

10056 

10000 

******* 

9997 

12.0 

47475,40 

11.9534 

47466.95 

* 

******* , 

11 .9S04 

47466.88 i 

10039 

10000 ! 

******* 

9997 

14.0 

40946.09 

1 13.9603 

40938. B0 

• 

* 

******* ! 

! 13.9574 

40938.76 1 

10029 

10000 

******* 

9998 

16.0 

36021.18 

15.9655 

36014.77 

• 

• 

^ ^ 'r o' ^ T | 

15.9623 

36014.74 i 

10023 

mmtmm 

******* 

9998 

18.0 

32181.73 

! 17.9695 

32176.01 

• 

******* J 

1 ‘ 17.9671 

32175.99 J 

10018 

10000 

******* 

9999 

20 . 0 

29109. 87 

19.9728 

29104.69 

i 

* 

******* J 

19.9706 

29104.68 ! 

10015 


******* 

9999 

25,0 

23600.08 

! 24.9789 

23595.89 

< 

■ 

******* j 

! 24.9770 

23595.88 ! 

10010 

10000 

******* 

9999 

30 . 0 

19966 . 96 

29.9830 

19963.41 

• 

• 

******* J 

29.9814 

19963.41 i 

10007 

.■;-v 

******* 

9999 

35.0 

174 IS . 81 

I 34.9860 

17412.72 

• 

• 

******* | 

! 34.9846 

17412.71 t 

10005 


******* 

10000 

40.0 

15546.43 

39.9884 

15543.67 

« 

i 

******* { 

39.9872 

15543.67 ! 

10004 


******* 

10000 

45.0 

14135.92 

1 44.9903 

14133.41 

i 

******* : 

! 44,9892 

14133.41 5 

10003 

h&B 

******* 

1 0 0 0 0 

50 . 0 

13050 . 73 

49.9919 

13048.42 

■ 

i 

******* j 

49.9910 

13048.41 i 

10003 

1 0000 

******* 

10000 

60.0 

11546.61 

! 59.9945 

11544.57 

• 

i 

******* : 

! 59.9938 

11544.56 ! 

• 10002 

10000 

******* 

10000 

70.0 

10642.75 

69.9965 

10640.86 

i 

i 

******* j 

, 69.9961 

10640.85 ! 

10002 

10000 


100 00 

80.0 

10155,80 

I 79.9983 

10154.01 

s 

******* : 

1 79.9981 

10154.00 1 

10002 

10000 

******* 

10 000 

90.0 

10001.72 

90.0000 

1 

9999.95 

1 

1 

******* i 

i 90.0000 
1 

9999.94 1 
« 
t 

10002 1 

10000 

- 

******* 

10000 

1 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 






COMPARISON OF GRADIENT ( GMD > , WHITE SANDS (CYBER > AND SPHERICAL SLAB (VAR I AN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM-MOIST MORNING SEA LEVEL INDEX OF REFRACTION: .0003661 SCALE HEIGHT: 5977.06 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYDER ! 

SPHERICAL SLAB ! 

ALTITUDE 

COMPUTED BY 

■ — — ■■ ■- 1 — " - — 

EACH SOLUTION METHOD 



E HEAS 

R MEAS 1 

E COR 

R COR 

E COR : 

E COR 

R COR i 

NO CORR 

! 


GMD H 

1 


CYB H 

• 

1 

VAR H 

DEG 

METER 

DEG 

METER 

DEG 

DEG 

METER J 

METER 



METER 



METER 

i 

METER 

0.0 

587895.64 

-.6874 

587780.17 

******* j 

-.5921 

i 

587812 . 09 1 

27037 



20000 



******* 

• 

i 

20978 

.5 

508308.90 

-.0244 

508222.99 

******* 

-.0288 

508226.08 1 

24643 



20000 



******* 


19962 

i.O 

445557.95 

. S7S4 

445489.43 

******* 

.5707 

445491.91 ! 

23296 



20000 



******* 


19964 

i .5 

394006.21 

1.1440 

393949.40 

******* 

i . 1429 

393952.49 ! 

22444 



20000 



******* 


19993 

2.0 

350965.81 

1 .6947 

350917.50 

******* 

1.6943 

350920.72 ! 

21867 



20000 



******* 


19998 

2.5 

314761.72 

2.2338 

314719.85 

******* 

2.2322 

314722.86 t 

21460 



20000 



******* 


19992 

3.0 

284143.21 

2.7647 

284106.38 

******* 

2.7620 

284109.02 1 

21165 



20000 



^ ^ ^ 

1 

19987 

3.5 

258112,04 

3.2898 

258079.24 

******* 

3.2876 

258081.51 ! 

20946 



20000 



******* 


19991 

4.0 

235854.91 

3.8104 

235825.39 

******* 

3.8116 

235827.35 ! 

20779 



20000 



******* 


20005 

4.5 

216710.82 

4.3276 

216684.02 

******* 

4.3351 

216685.74 { 

20651 



20000 



******* 


20029 

5.0 

200141 .79 

4.8421 

200117.2? 

******* 

4.8522 

200118.80 1 

20551 



20000 



******* 

1 

20035 

6.0 

173070 .17 

5.86S3 

173049.27 

******* 

5.8720 

173050.53 l 

20406 



20000 



******* 


20020 

7.0 

152050.76 

6.8828 

152032,58 

******* 

6.8875 

152033 .71 ! 

20310 



20000 



******* 


20013 

8.0 

135368.23 

7.8965 

135352.15 

******* j 

7.9000 

135353.22 i 

20244 



20000 



******* 


20008 

9.0 

121866. 12 

8.9075 

121851.71 

******* 1 

8.9102 

121852.72 5 

20196 



20000 



******* 


20006 

10.0 

110749.42 

9,9164 

110736,38 

******* 1 

9.9186 

110737.33 1 

20161 



20000 



******* 

I 

20004 

12,0 

93591 . 13 

11.9302 

93580.16 

******* J 

11.9317 

93581.02 1 

20114 



20000 



******* 

1 

1 

20003 

14.0 

81022.63 

13.9402 

81013,16 

******* 

13.9413 

31013.93 i 

20034 



20000 



******* 


20002 

16.0 

71455.96 

15,9479 

71447.63 

******* j 

15.9488 

71448.32 ! 

20065 



20000 



******* 

j 

20001 

18.0 

63951 . 13 

17,9540 

63943.69 

******* 

17.9546 

63944.31 ! 

20051 



20000 



******* 


20001 

20 . 0 

57920 . 05 

19.9588 

57913.31 

******* j 

19.9594 

57913.89 i 

20041 



20000 



******* 

1 

r 

20001 

25.0 

47049.71 

24.9679 

47044.25 

******* : 

24.9682 

47044.72 ! 

20026 



20000 



******* 


200 0 0 

30 . 0 

39849.62 

29.9741 

39844.99 

******* 

29.9743 

39845.40 1 

20018 



20000 



******* 


20000 

35 . 0 

34780,87 

34.9786 

34776.83 

******* 

34.9787 

34777,19 ! 

20013 



20000 



******* 


20000 

40.0 

31060 . 73 

39,9822 

31057,13 

******* 

39.9823 

31057.45 { 

20010 



20000 




j 

20000 

45 . 0 

28250.77 

44,9851 

28247.50 

******* 

44.9351 

28247.79 ! 

20007 



20000 



******* 


2 0 0 0 0 

50 . 0 

26087 .23 

49.9875 

26084.21 

******* 

49.9875 

26084.48 ! 

20006 



20000 





20000 

60 . 0 

23086.63 

59.9915 

23083.96 

******* 

59.9914 

23084.19 ! 

20004 



20000 



******* 


20000 

70 . 0 

21282.33 

69.9946 

21279.87 

******* 

69.9946 

21280.03 1 

20003 



20000 





20000 

80 . 0 

20310 . 04 

79.9974 

20307,69 ! 

******* 

79.9974 

20307.90 ! 

20002 



20000 



******* 


20000 

90 . 0 

20002.31 1 

90.0000 

i 

19999.99 1 

******* 

! 

90.0000 

20000.20 l 
» 
* 

20002 


i 

20000 


i 

******* 


20000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



BASED OH 
WEATHER 

COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB (VAR IAN) CORRECTION METHODS 
RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 
DATA FOR TYPICAL WARM-MOIST MORNING SEA LEVEL INDEX OF REFRACTION: .0003661 SCALE HEIGHT: 5977.0( 

75-FH-60 

METERS 

! INPUT 

DATA 

1 GMD GRAD 

REF SOL 

: CYBER ! 

SPHERICAL SLAB J 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 2 

! E MEAS 

R MEAS 

1 E COR 

R COR 

! E COR 1 

E COR 

R COR 

NO CORR 

GMD H 

CYB H 

VAR H 

2 DEG* 

METER 

! ’ DEG 

METER 

1 DEG 2 

DEG 

METER 2 

METER 

METER 

METER 

METER 

{ 0.0 

895393.45 

! -.808? 

895272.60 

1 i 

l ******* i 

-.7212 

2 

895309.54 J 

62543 

50000 


51362 1 

* .5 

814515.39 

1 -.1268 

81442S.79 

; ******* i 

-.1698 

814432.15 2 

58845 

50000 

******* 

49393 

! i .0 

748004.85 

1 .4858 

747933.48 

1 ******* ; 

.4607 

747935.44 2 

56664 

soooo 

******* 

49674 

1 1.5 

690508.53 

: 1.0641 

690449.36 

: ******* ; 

1 . 0543 

690452.51 ! 

55214 

50000 

******* 

49881 

! 2.0 

639746.99 

} 1.6227 

639696.64 

S ******* j 

1.6203 

639700.16 2 

54181 

50000 

******* 

49972 

i 2,5 

594501.71 

1 2.1683 

594458.04 

j ******* j 

2.1685 

594461.43 2 

53415 

500 00 

******* 

50 000 

! 3,0 

553967 . 63 

2 2.7049 

553929.17 

i ******* i 

2,7061 

553932.22 ! 

52831 

50000 

******* 

50010 

3.5 

517537.31 

1 3.2347 

517503.02 

i ******* i 

3.2380 

517505.70 1 

52377 

500 00 

******* 

50029 

i 4,0 

484713.82 

1 3.7594 

484682.94 

s ******* : 

3.7676 

484685.28 2 

52018 

50000 

******* 

50068 

O 1 4 - 5 

455080.59 

i 4.2803 

455052.53 

i ******* j 

4.2967 

455054.61 2 

51730 

50000 

******* 

50129 

1 2 5.0 

428263.03 

i 4.7980 

428237.34 

i ******* j 

4.8177 

428239.21 2 

51496 

50000 

******* 

50145 

i ; 6,o 

381870.44 

! 5.8265 

381848.50 

j ******* i 

5.8421 

381850.09 2 

51143 

500 00 

******* 

50101 

i 7.0 

343430 .44 

1 6.8484 

343411.33 

2 ******* { 

6.8613 

343412.76 2 

50897 

50000 

******* 

50075 

! 8.0 

311296.02 

2 7.8656 

311279.11 

2 ******* ; 

7. 8767 

311280.45 ! 

50719 

50000 

******* 

50058 

! 9.0 

284190.37 

! 8.8795 

284175.21 

2 ******* 2 

8.8892 

284176.48 2 

50588 

50000 

******* 

50046 

! 10.0 

261121.44 

! 9.8909 

261107.70 

{ ******* 2 

9.8995 

261108.91 2 

50489 

50000 

******* 

50039 

i 12.0 

224173.40 

i 11.9085 

224161.84 

I ******* 2 

11.9156 

224162.91 2 

50349 

50000 

****** * 

50026 

I 14.0 

196107.96 

! 13.9215 

196097.97 

2 ******* ; 

13.9275 

196098.94 2 

50260 : 

50000 

******* 

50 018 

! 16.0 

174207.80 

i 15.9315 

174199.01 

« ******* 2 

15.9367 

174199.87 2 

50200 ; 

50000 

******* 

50 0 14 

! 18.0 

156721 .01 

! 17.9393 

156713.16 

{ ******* J 

17.9439 

1S6713.95 2 

50158 1 

50000 

******* 

50011 

! 20.0 ' 

142485.38 

! 19.9457 

142478,27 

1 * ^ ^ £###£ I 

19.9498 

142478.99 J 

50127 : 

50000 

******* 

50 009 

{ 25.0 

116449.21 

i 24.9575 

116443,45 

2 ******* 2 

24.9607 

116444.04 2 

50080 i 

50000 

******* 

50006 

: 3o.o 

98962.82 

1 29.9657 

98957.94 

2 ******* J 

29.9682 

98950.45 2 

50053 ! 

50000 

******* 

50003 1 

! 35.0 

86554 .10 

i 34.9716 

86549.83 

2 ******* ; 

34.9737 

86550.28 5 

50036 ! 

50000 

******* 

50 002 

S 40.0 

77400 .92 

i 39.9763 

77397,11 

2 ******* ; 

39.9780 

77397.51 2 

50026 

50000 

******* 

50 0 0 1 

! 45.0 

70463.99 

1 44.9801 

70460.53 

I ******* ; 

44.9816 

70460.90 5 

50019 ! 

50000 

******* 

50 001 

1 50.0 

65110 .44 

J 49.9834 

65107,25 

2 ******* 2 

49.9845 

65107.58 2 

50014 

50000 

******* 

50000 

1 60.0 

57669 .22 

1 59.9886 

57666.40 

2 ******* 2 

59.9893 

57666.69 1 

50008 i 

50000 

******* 

50000 

1 70.0 

53186.28 

J 69.9928 

53183.69 

J ******* j 

69.9933 

53183,96 2 

50004 

50000 

******* 

50000 

J 80.0 

50760 . 16 

\ 79.9965 

50765.68 

2 ******* ; 

79.9967 

50765.94 ! 

50003 1 

50000 

******* 

50000 

S 90,0 

1 

1 

50002.45 

1 90.0000 

i 

50000.00 

2 *******' 2 
2 2 

90.0000 

50000.26 J 

t 

1 

50002 

50000 

, 

50 000 

***** INDICATES INPUT 

VALUES HAVE 

EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 





COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER) AND SPHERICAL SLAB (VARIANT CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM-MOIST MORNING SEA LEVEL INDEX OF REFRACTION I .0003661 SCALE HEIGHT s 5977.06 METERS 


INPUT 

DATA 

GMD GRAD 

REF SOL 

CYBER 

i SPHERICAL SLAB 


ALTITUDE 

COMPUTED BY 

EACH SOLUTION 

METHOD 

E MEAS ' 

R MEAS 

* E COR 

R COR 

E COR 

l E COR 

R COR 


NO CORR 



GMD H 



CYB H 

* 

VAR H 

DEG 

METER 1 

DEG 

METER 

DEG 

1 DEG 

METER 


METER 



METER 



METER 

J 

METER 

0.0 

1235378.72 

-.8736 

1235255.79 

******* 

! 

S -.7919 

1235294*. 66 


118S38 



100000 



******* 

1 

1 

s 

101739 

.5 

1153895.09 l 

-.1777 

1153804.39 J 

******* 

i - . 245S 

1153811.80 


113438 



100000 

1 


******* 

i 

• 

98657 

1.0 

1085594.94 i 

.4438 

1085522.92 i 

******* 

! .4051 

1085524.03 


110378 



100000 

! 


******* 

• 

* 

99277 

1.5 

1025191.00 

1 . 0285 

1025131.42 

******* 

i 1.0120 

1025134.16 


108309 



100000 



******* 

i 

99708 

2.0 

970466.99 

1.5919 

970416.37 

******* 

I 1.5867 

970419.69 


106805 



100000 



******* 

• 

i 

99911 

2.5 

920359.09 

2.1413 

920315.23 

******* 

i 2.1408 

920318.52 ' 


105673 



100000 



******* 

• 

• 

99992 

3.0 

874154,51 

2.6808 

874115.92 J 

******* 

! 2 . 6825 

874118.91 


104792 



100000 



******* 

i 

100025 

■ 3.5 

831413.12 

3.2131 

831378,74 

******* 

r 3.2178 

831381.37 


104096 



100000 



******* 

• 

i 

100066 

4.0 

791774.05 1 

3.7398 

791743.09 1 

******* 

1 3.7502 

791745.41 


103536 



100000 

1 


******* 

i 

i 

100140 

4.5 

754948.53 

4.2623 

754920.41 

******* 

i 4.2818 

754922.48 


103079 



100000 



******* 

i 

100252 

5.0 

720679.43 1 

4.7815 

720653.70 l 

******* 

* 4.8046 

720655.56 


102700 

i 


100000 



******* 

• 

» 

100284 

6.0 

659029.73 

5.8122 

659007.77 

******* 

! 5.8310 

659009.34 


102117 



100000 



******* 

i 

i 

100209 

7.0 

605388.02 ! 

6.8358 

605368.89 

******* 

I 6.8S17 

605370.32 


101698 



100000 



******* 

* 

• 

100162 

8.0 

558555.31' I 

7.8544 

558538.38 1 

1 ******* 

! 7.8682 

558539.72 


101387 



100000 



******* 

• 

• 

100131 

9.0 

517509.69 ! 

8.8694 

517494.52 I 

******* 

1 8.8816 

517495.79 


101149 

1 


100000 



******* 

i 

i 

100106 

10. 0 

481418.14 I 

i 9.8817 

481404.39 I 

: ******* 

\ 9.8927 

481405,59 


100967 



100000 



******* 

i 

i 

100089 

12,0 

421269.81 i 

11.9008 

421258.25 J 

******* 

! 11.9099 

421259.32 


100704 



100000 

1 

1 


******* 

i 

i 

100063 

14.0 

373602.20 i 

1 13.9148 

373592.22 ! 

! ******* 

! 13.9226 

373593.18 


100532 



100000 



******* 

i 

« 

100047 

16,0 

335206.20 1 

15.9256 

335197.41 ! 

******* 

} 15.9324 

335198.28 


100413 



100000 

t 


******* 

i 

i 

100036 

18.0 

303812.16 ! 

i 17.9341 

303804.30 ! 

| ******* 

! 17.9401 

303805.08 

• 

■ 

100330 



100000 


• 

******* 

i 

i 

100030 

20.0 

277777.14 ! 

19,9410 

277770.03 1 

! ******* 

* 19.9464 

277770.75 


100266 



100000 



******* 

i 

i 

100023 

25.0 

229135.95 ! 

! 24.9538 

229130.18 

1 if ^ ^ Y' ^ 

1 V n' ^ ^ * 

i 24.9580 

229130.78 

• 

i 

100166 



1 00000 


1 

******* 

i 

i 

100014 

30 . 0 

195781.76 ! 

29.9626 

195776,88 ! 

: ******* 

! 29.9660 

195777.39 


100110 



100000 



******* 

i 

I 

100009 

35 . 0 

171811.80 ! 

! 34.9691 

171807. S4 ! 

i ******* 

J 34.9719 

171807.98 

» 

i 

100076 



1 0 0 0 0 0 



******* 

i 

1 0 0 0 06 

40 , 0 

153985.23 ! 

39.9742 

153981.42 ! 

******* 

1 39.9766 

153981 ,82 


100054 



100000 



******* 

i 

100004 

45.0 

140399.90 ! 

1 44.9784 

140396.44 ! 

i ******* 

! 44.9803 

140396.80 

i 

■ 

100038 



1 00000 


1 

^ 2|C 

» 

100 003 

50 . 0 

129875.68 ! 

49.9819 

129872.49 i 

******* 

*, 49.9835 

129872.83 


100028 



1 ooooo 



******* 

i 

1 00003 

60.0 

115191.12 ! 

! 59.9875 

115188.30 ! 

i ******* 

! 59.9886 

115188.59 


100014 



100000 


1 

******* 

• 

i 00 0 0 1 

70 . 0 

106316.83 i 

69.9921 

106314.23 ! 

******* 

1 69.9928 

106314.50 


100007 



100000 



******* 

i 

i 0 0 0 0 1 

80.0 

101521.90 ! 

i 79.9962 

101519.42 ! 

; ******* 

i 79.9965 

101519.68 

• 

100004 



100000 


1 

******* 

« 

10 0 0 0 0 

90 . 0 

100002.45 \ 

90.0000 

i • 

looooo.oo ; 

******* 

> 

i 90.0000 

100000.26 


100002 



100000 



******* 

i 

i 

100000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



APPENDIX H 

TABULATED REFRACTION COMPARISON DATA 
FOR EDWARDS TYPICAL WARM DAY AFTERNOON 


H-l 



TABLE H-I 


WEATHER DATA FOR TYPICAL WARM DAY AFTERNOON 


ALTITUDE 

(FT) 

TEMPERATURE 
(DEG C) 

DEW POINT 
(DEG C) 

PRESSURE 

(MB) 

N X 1 0E6 

2316 

34.00000 

1.84997 

929.00000 

262.69745 

3000 

32.00000 

-.15002 

903.26001 

254.25128 

4000 

29.00000 

-4.15002 

891.03003 

247.47284 

5000 

26.00000 

-8.15002 

857.80005 

236.51797 

6000 

23.00000 

-12.15002 

819.53003 

225.15683 

7000 

20.00000 

-16.15002 

794.59998 

218.00351 

8000 

17.00000 

-20.15002 

768.16003 

211.01862 

9000 

14.50000 

-21.15002 

739.39001 

204.66995 

10000 

12.00000 

-22.15002 

713.06006 

198.90247 

11000 

9.50000 

-23.15002 

688.07996 

193.430S7 

12000 

7.00000 

-25.15002 

663.22998 

187.56149 

13000 

4.50000 

-27.15002 

639.75000 

182.07138 

14000 

2.00001 

-29.15002 

617.23010 

176.84204 

15000 

-.50000 

-31.15002 

594,35999 

171.49615 

16000 

-3.00000 

-33.15002 

572.38000 

166.37622 

17000 

-5.50000 

-36.15002 

551.87000 

161.49234 

18000 

-8,00000 

-39.15002 

531.03003 

156.53363 

19000 

-10.50000 

-42.15002 

510.69000 

151.71918 

20000 

-13.00000 

-45.15002 

491.85999 

147.33475 

21000 

-15.70001 

-48.15002 

473.03998 

143.03632 

22000 

-18.40002 

-51. 15002 

454.40997 

138.74869 

23000 

-21.30002 

-54.15002 

437.02002 

134.89230 

24000 

-24.00003 

-57.15002 

420 . 14001 

131.02570 

25000 

-26.70001 

-60.15002 

403.20001 

127.07535 

26000 

-29,50003 

-64.15002 

387.12000 

123.36671 

27000 

-32.30000 

-68,14999 

372.11011 

119.93466 

28000 

-35.10002 

-72.15001 

356.70007 

116.30334 

29000 

-37.90002 

-76.15002 

341.96997 

112.81746 

30000 

-40.70001 

-80 , 15002 

328.31000 

109.60959 

31000 

-40.67321 

-78.92619 

328.18799 

109.49875 

32000 

-43.26289 

-78.77321 

315.28833 

106.24097 

33000 

-47.19596 

-79.23215 

296,06165 

101.44191 

34000 

-51 . 19937 

-79.84407 

276.95868 

96.70723 

35000 

-54.00003 

-80.15002 

264.42999 

93.64259 

36000 

-56.30975 

-80 . 15002 

255.89999 

91.58688 

37000 

-59.35925 

-80.15002 

245.22552 

88.96097 

38000 

-62.65390 

-80.15002 

233.50607 

85.99347 

39000 

-65.69904 

-80,15002 

221.84106 

82.91298 

40000 

-68.00003 

-80 . 15002 

211.32999 

79.94809 

41000 

-69.18581 

-80 . 15002 

201.81140 

76.86771 

42000 

-69.44051 

-80.15002 

192.46701 

73,42751 

43000 

-69.10233 

-80 . 15002 

183.41089 

69.83629 

44000 

-68.50945 

-80 . 15002 

174,75720 

66.30289 

45000 

-68.00003 

-80.15002 

166.62000 

63,03609 

46000 

-67.35915 

-80.15002 

158.83591 

59.92341 

47000 

-66.26350 

-80.15002 

151,21689 

56.76656 

48000 

-64.88873 

-80,15002 

143.84003 

53,64563 

49000 

-63.41046 

-80.15002 

136.78241 

50.64070 

50000 

-62. 00433 

-80 . 15002 

130 . 12112 

47.83187 


H-2 



TABLE H-I 


Continued 



WEATHER DATA 

FOR TYPICAL 

WARM DAY AFTERNOON 


ALTITUDE 

TEMPERATURE 

DEW POINT 

PRESSURE 

N X 1 0E6 

(FT) 

(DEG C) 

(DEG C) 

(MB) 


51000 

-60.56895 

-80.15002 

123.71893 

45.17155 

52000 

-58.98966 

-80.15002 

117.44495 

42.58508 

530 0 0 

-57,42764 

-80.15002 

iii. 38992 

40.11436 

54000 

-56.04404 

-80,15002 

105.64471 

37.80131 

55000 

-55.00003 

-80.15002 

i00. 30000 

35.68780 

56000 

-54.47746 

-80.15002 

95.31410 

33.79884 

57000 

-54.41688 

-80.15002 

90.58319 

32.10184 

58000 

-54.61759 

-80.15002 

86.10529 

30.55256 

59000 

-54.87888 

-80.15002 

81.87820 

29.10661 

60000 

-55.00003 

-80.15002 

77.89999 

27,71970 

61000 

-54.98045 

-80.15002 

74.13089 

26,37768 

62000 

-54.96028 

-80.15002 

70,53699 

25.09793 

63000 

-54.94990 

-80.15002 

67.12457 

23.88339 

64000 

-54.95969 

-80,15002 

63,90015 

22.73705 

65000 

-55.00003 

-80,15002 

60.87000 

21.66182 

66000 

-55.07311 

-80.15002 

58.00195 

20,64641 

67000 

-55.14839 

-80.15002 

55.26201 

19,67670 

68000 

-55.18713 

-80.15002 

52.65412 

18.75158 

69000 

-55.15058 

-80.15002 

50.18214 

17.86984 

70000 

-55.00003 

-80.15002 

47,85000 

17,03036 

71000 

-54.62611 

-80.15002 

45.62428 

16.21456 

72000 

-54,02283 

-80,15002 

43.47842 

15.41321 

73000 

-53.30652 

-80.15002 

41.42638 

14.63907 

74000 

-52.59348 

-80.15002 

39.48225 

13.90495 

75000 

-52.00003 

-80,15002 

37.66000 

13.22361 

76000 

-51.53118 

-80.15002 

35.93859 

12.59022 

77000 

-51.10735 

-80.15002 

34.29008 

11.99045 

78000 

-50,71793 

-80.15002 

32,71828 

11,42289 

79000 

-50.35236 

-80,15002 

31.22698 

10.88610 

80000 

-50.00003 

-80.15002 

29.82000 

10.37866 

81000 

-49.62761 

-80,15002 

28.47564 

9.89296 

82000 

-49,20585 

-80.15002 

27,17604 

9.42399 

83000 

-48.73029 

-80.15002 

25.93060 

8,97419 

84000 

-48.19650 

-80.15002 

24.74878 

8.54601 

85000 

-47,60004 

-80.15002 

23,64000 

8.14192 

86000 

-46.86880 

-80.15002 

22.58916 

7.75574 

87000 

-46.01745 

-80.15002 

21.57758 

7.38245 

88000 

-45.17173 

-80.15002 

20.60943 

7.02622 

89000 

-44.45734 

-80.15002 

19.68884 

6.69122 

90000 

-44.00003 

-80.15002 

18.82000 

6.38161 

91000 

-43.87155 

-80.15002 

18.02932 

6.10960 

92000 

-43.63983 

-80.15002 

17.15881 

5.81004 

93000 

-42.94996 

-80.15002 

16.22529 

5.47901 

94000 

-42.11295 

-80.15002 

15.44332 

5.19646 

95000 

-41.53740 

-80 . 15002 

14.95721 

5.01962 

96000 

-41 , 17400 

-80.15002 

14.63868 

4.90530 

97000 

-40.48238 

-80.15002 

14.12607 

4.72159 

98000 

-39.56448 

-80,15002 

13.48358 

4.49147 

99000 

-38.53429 

-80 . 15002 

12.78299 

4.24069 

100000 

-37.50578 

-80.15002 

12.09601 

3.99496 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYBER > AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
• BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM DAY AFTERNOON SEA LEVEL INDEX OF REFRACTION! .0002866 SCALE HEIGHT! 7406.42 METERS 


INPUT 

DATA 1 

GMD GRAD 

REF SOL 

1 CYBER \ 

SPHERICAL SLAB I 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

E HEAS 
DEG 

R HEAS 1 

METER ‘ ! 

E COR 
DEG 

R COR 
METER 

l E COR i 
I DEG 1 

E COR 
DEG 

R COR I 

METER 1 

NO CORR 
METER 

t 

GMD H 
METER 

CYB H : 

METER 

VAR H 
METER 

0.0 

< 

129606.46 1 

-.1397 

129570.78 

1 1 
1 -.1487 ! 

-.1420 

\ 

129563.38 1 

1317 


1000 


980 

995 

.5 

75503.79 1 

.4201 

75483.35 

! .4172 ! 

.4172 

75482.15 l 

1106 


1000 


997 

996 

i.O 

49040.63 1 

.9486 

49027.43 

1 .9469 ! 

.9488 

49027.15 J 

1044 


1000 


999 

1000 

1 , 5 

35341.99 i 

1.4632 

35332.49 

S 1.4619 

1.4638 

35332.44 1 

• 1023 


1000 


1000 

1000 

2.0 

27368.69 1 

1.9717 

27361.34 

J 1.9706 1 

1.9719 

27361.35 ! 

1014 


1000 


1000 

1000 

2.5 

22247.38 1 

2.4771 

22241.41 

1 2.4761 5 

2.4769 

22241.44 1 

1009 


1000 


1000 

1000 

3.0 

18708.64 ! 

2.9809 

18703.62 

1 2.9799 1 

2.9803 

18703.64 > 

1006 


1000 


1000 

1000 

3.5 

16127.28 * 

3.4836 

16122.96 

1 3.4827 i 

3.4830 

16122.98 i 

1005 


1000 


1000 

1000 

4.0 

1416S.40 1 

3.9857 

14161.60 

< 3.9848 l 

3.9852 

14161.61 { 

1004 


1000 


1000 

1000 

4.5 

12625,89 

* 4.4874 

12622.50 

1 ' 4.4865 i 

4.4874 

12622.51 ! 

1003 


1000 


1000 

1000 

5.0 

11386.63 ! 

4.9887 

11383.58 

J 4.9878 ! 

4.9882 

11333.58 1 

1002 


1000 


1000 

1000 

*6;o 

9516.76 

5.9907 

9514.20 

1 S . 9898 1 

5.9901 

9514.21 { 

1002 

\ 

1000 


1000 

1000 

7.0 

8174.55 1 

6.9921 

8172.36 

J 6.9913 J 

6.9915 

8172.36 ! 

1001 

! 

1000 


1000 

1000 

! 8.0 

* 7164.81- 1 

7,9932* 

7162,89 

1 7.9924 J 

7.9925 

7162.89 { 

1001 


1000 

1 

1000 

1000 

1 9.0 

6378.38 I 

8.9940 

6376.67 

\ 8.9932 J 

8.9934 

6376.67 1 

1001 


1000 


1000 

1000 

1 10. 0 

5748.75 1 

! 9.9947 

5747.21 

1 9.9939 ! 

9.9940 

5747.21 1 

1001 


1000 

• 

1000 

1000 

12.0 

4804,37 ! 

11.9957 

4803.09 

J 11.9950 S 

11.9950 

4803.09 I 

1001 

1 

1000 

i 

1000 

1000 

14,0 

4130,48 ! 

! 13.9964 

4129.37 

1 13.9957 1 

1 13.9958 

4129.37 { 

1001 


1000 

i 

i 

1000 

1000 

16.0 

3626.06 1 

15.9970 

3625.09 

\ 15.9963 S 

15.9963 

3625.09 l 

1000 


1000 

1 

1000 : 

1000 

18.0 

3234.84 i 

t 17,9974 

3233.97 

i 17.9967 I 

! 17.9967 

3233.97 ! 

1 Q 0 0 


1000 

i 

1000 

1000 

20.0 

2923.11 ! 

19.9977 

2922.32 

i 19.9971 I 

19.9971 

2922.32 ! 

1000 


1000 

1 

1000 

1000 

25,0 * 

2366.03 ! 

1 24.9983 

* 2365.40 

i 24.9977 i 

! 24.9977 

2365.40 i 

* 1000 


1000 

1 

1 

1000 

1000 

1 30.0 

1999.97 1 

29.9988 

1999.44 

! 29.9981 ! 

29.9982 

1999.44 J 

1000 


1000 

9 

1 

1000 

1000 

1 35.0 

1743.57 1 

1 34.9990 

1743.10 

I 34.9985 1 

1 34.9985 

1743.10 1 

1000 


1000 

1 

1000 

1000 

I 40,0 

1555.94 1 

! 39.9992 

1555.52 

S 39.9987 J 

39.9987 

1555.52 ! 

1000 


1000 

1 

1 

1000 

1000 

45.0 

1414.48 ! 

1 44.9994 

• 1414.10 

1 44,9989 ! 

44.9989 

1414.10 ; 

1000 


1000 

1 

1 

1000 

1000 

50.0 

1305.53 1 

! 49.9996 

1305.18 

1 49.9991 J 

49.9991 

1305.18 ! 

1000 


1000 

1 

1000 

1000 

60.0 

1154.86 ! 

1 59.9997 

1154.55 

S 59,9994 ! 

59.9994 

1154.55 1 

1000 


1000 

ft 

1000 

1000 

70.0 

1064. 3S I 

i 69.9998 

1064.07 

i 69.9996 ! 

69.9996 

1064.07 S 

1000 


1000 

1 

1000 

1000 

80.0 

1015.61 

l 79.9999 

10 IS , 34 

t 79.9998 

79.9998 

1015.34 ! 

1000 


1000 


1000 

1000 

1 90.0 

1000.19 ! 

t 90.0000 

999.92 

1 90.0000 1 

I 90.0000 

999.92 i 

1000 


1000 

ft 

ft 

1000 

1000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD) , WHITE SANDS (CYDER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 

BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 

WEATHER DATA FOR TYPICAL WARM DAY AFTERNOON SEA LEVEL INDEX OF REFRACTION: .0002866 SCALE HEIGHT: 7406.42 METERS 

— _ — — — _ — — — — ______ . . ■ _ _ 

INPUT DATA 1 GMD GRAD REF SQL 1 CYDER ! SPHERICAL SLAB 1 ALTITUDE COMPUTED BY EACH SOLUTION METHOD 

E MEAS R MEAS l E COR R COR \ E COR \ E COR R COR i NO CQRR \ GMD H \ CYB H \ VAR H 

DEG METER I DEG METER \ DEG 1 DEG * METER S METER I METER I METER S METER 


0,0 . 

182767.91 I 

-.1935 

182719.38 i 

-.2097 1 

-.1957 

• ,s 

123608.97 

.3724 

* 123577.03* l 

,3681 : 

.3667 

1 . 0 

87856.95 1 

.9104 

87834.49 1 

.9090 i 

.9116 

i.s 

*66256.26 

* 1.4329 

66239.42 1 

* 1 . 4322 ( 

1 . 4350 

2.0 

52525.67 1 

1.9470 

52512.35 l 

1,9466 ! 

1.9484 

2.5 

43258.71 

2.4566 

43247.76 i 

2.4562 i 

2.4570 

3.0 

36664.92 1 

2.9633 

36655.63 i 

2.9630 ! 

2.9631 

3', S 

* 3176S.63 

3.4683 

* 31757,60 i 

3.4680 i 

3.4679 

4.0 

27996. SO I 

3.9722 

27989.41 J 

3.9718 i 

3.9721 

4.5 * 

25013.54 

4.4752 

. 25007.22’ i 

* 4.4748' 1 

4.4760 

5,0 

22597,94 1 

4.9777 

22592.23 1 

4.9773 1 

4.9779 

6.0 

18930.73 ! 

5.9815 

18925.94 1 

5.9810 I 

5.9813 

7.0 

16283,64 i 

6.9842 

16279.53 ! 

6.9837 i 

6,9839 

8.0 * 

14285.94 I 

7.9863 

• 14282.33 1 

7.98S7 i 

7.9858 

9,0 

12725.89 J 

8. 9879 

12722.68 I 

8.9873 1 

8.9874 

10. 0 ’ 

11475.00 : 

* 9.9891 

11472.11 ! 

9.9886 : 

9.9886 

12.0 

9595.69 1 

11.9911 

9593.26 ! 

11.9905 : 

11.9905 

14.0 

8252.76 ! 

13,9925 

8250.67 1 

13.9919 1 

13.9919 

16.0 

7246,68 ! 

15.9935 

7244.84 1 

15.9930 ! 

15.9929 

18.0 

6466.04 ! 

17.9944 

6464.41 I 

17.9938 i 

17.9938 

20.0 

5843.46 J 

19.9950 

5841.99 l 

19.9945 1 

19.9944 

25.0 

4730.7V I 

24.9962 

4729.51 \ 

24.9957 J 

24,9957 

30,0 

3999.35 J 

29.9970 

3998.34 J 

29.9965 : 

29.9965 

35.0 

3486.84 1 

34.9976 

3485.96 1 

34.9971 ! 

34.9971 

40.0 

3111.57 ! 

39.9981 

3110.79 * 

39.9976 i 

39.9976 

45.0 

2828.75 1 

44.9984 

2828.03 \ 

*44.9980 ! 

44.9980 

50.0 

2611.10 1 

49.9987 

2610.44 1 

49.9983 ! 

49.9983 

60.0 

2309.82 1 

59.9991 

2309.24 » 

59.9988 \ 

59.9988 

70.0 

2128,84 i 

69.9995 

2128.30 ! 

69,9993 I 

69.9993 

80.0 

2031.20 1 

t 79.9998 

2030.69 ! 

79.9996 i 

79.9996 

90.0 

2000.36 i 

90.0000 

1999.85 i 

90.0000 : 

90.0000 


* * 


\ 


182715.0 7 1 

2618 

2000 : 

1949 i 

1993 

123575.27 1 

2276 

2000 1 

1992 « 

1988 

87834.05 { 

2138 

2000 : 

1999 \ 

2002 

66239.41 ! 

2078 

2000 S 

2000 \ 

2003 

S2S12.47 J 

2049 

2000 : 

2000 1 

2001 

43247.89 ! 

2033 

2000 : 

2000 » 

2000 

36655.76 ! 

2024 

2000 i 

2000 1 

2000 

31757.70 l 

2018 

2000 i 

2000 i 

2000 

27989.50 ! 

2014 

2000 i 

2000 : 

2000 

25007.28 i 

2011 

2000 : 

2000 i 

2000 

22592.28 i 

2009 

2000 } 

2000 : 

2000 

18925.97 ! 

2007 

2000 \ 

2000 S 

2000 

16279.55 ! 

2005 

2000 i 

2000 i 

2000 

14282.35 1 

2004 

2000 i 

2000 I 

2000 

12722.69 J 

2003 

2000 i 

2000 l 

2000 

11472.12 1 

2003 

2000 i 

■2000 \ 

2000 

9593.27 i 

2002 ! 

2000 1 

2000 : 

2000 

8250.68 S 

2002 

2000 i 

20 0 0 J 

2000 

7244.85 S 

2001 ! 

2000 { 

2000 ; 

2000 

6464,41 i 

2001 

2000 J 

2000 S 

2000 

5841.99 1 

2001 ! 

2000 } 

2000 ! 

2000 

4729.52 ! 

2001 

2000 : 

2000 i 

2000 

3998.34 i 

2001 ! 

2000 : 

2000 ! 

2000 

3435.97 i 

2001 

2000 S 

2000 J 

2000 

3110.79 i 

2001 I 

2000 S 

2000 : 

2000 

2828.04 S 

2001 

2000 \ 

20 0 0 i 

2000 

2610.45 1 

2000 ! 

2000 ; 

2000 : 

2000 

2309.24 ! 

2000 ! 

2000 S 

2000 } 

2000 

2128.30 * 

2000 i 

2000 : 

2000 l 

2000 

2030.69 ! 

2000 ! 

2000 : 

2000 i 

2000 

1999.85 S 

2000 i 

2000 S 

2000 i 

2000 


i i 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



BASED ON 
WEATHER 

COMPARISON OF GRADIENT ( GMD> , WHITE SANDS (CYBER) 
RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME 
DATA FOR TYPICAL WARM DAY AFTERNOON SEA LEVEL INDEX 

AND SPHERICAL SLAD ( VARIAN) CORRECTION METHODS 
MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 
OF REFRACTION! .0002866 SCALE HEIGHT! 7406. 42 

7S-FM-60 

METERS 

! INPUT 

DATA 

GMD GRAD 

REF SOL l 

CYBER 1 

SPHERICAL SLAB * 

ALTITUDE COMPUTED BY 

EACH 

SOLUTION METHOD ! 

! E MEAS 

R MEAS 

E COR 

R COR 1 

E COR 

E COR 

R COR ! 

NO CORR S 

GMD H 

CYB H « 

VAR H 

1 DEC 

MSTER 

1 DEG 

METER i 

DEG 

DEG 

METER 1 

METER ! 

METER 

METER ! 

METER 

1 

i o.o 

286733 . 95 

r 

\ -.2878 

i 

286664,61 ! 

-.3290 

-.2413 

1 

286668.78 ! 

6442 1 

5000 

I 

4797 

5233 

: .5 

222897.48 

1 .2851 

222846.01 1 

,2749 

.2860 

222845 ; 58 1 

5837 1 

5000 

t 

4962 

5003 

; l.o 

176325.55 

1 .8339 

176285 . 87 ! 

.8335 

.8389 

176286.34 J 

5512 ! 

5000 

i 

5000 

5015 

! I'.S 

142736 ;57 

1 1.3674 • 

142704.93 1 

■ 1.3702 

1.3736 

14270S , 86 1 

5331 1 

5000 

* 

• 

5008 

5015 

! 2.0 

118346.81 

1 1.8911 

118320.80 ! 

1 .8947 

1 .8959 

118321.83 I 

5226 1 

5000 

i 

5009 

5010 

I 2 . 5 

100301.38 

1 2.4082 

100279,45 

2.4119 

2.4112 

100280.43 1 

5162 • 

5000 

I 

SO 08 

5005 

i 3.0 

86632.45 

! 2.9211 

86613.57 i 

2.9246 

2.9226 

86614.42 ! 

5120 \ 

5000 

• 

. 

5007 

5002 

i • 3 . S 

76028.29- 

1 * 3.4310 ■ 

76011.76 

3.4343 

3.4319 

76012.49 

5093 l 

5000 

i 

5006 

500 1 

i 4.0 

67617.24 

3.9388 

67602.56 1 

3.9418 

3.9404 

67603.18 ! 

5073 ! 

5000 

1 

5005 

5002 

!* 4 . S ■ 

60812.17 

! 4.4451 

60798.99 

4.4478 

4.4484 

60799.51 1 

5059 } 

5000 

i 

5004 

5004 

ffi ! 5.0 

55209,80 

J 4.9503 

55197.84 1 

4.9528 

4 . 9S3S 

SS198.28 i 

5049 i 

5000 

1 

5004 

5003 

1 ! 6.0 

46559.91 

I 5.9583 

46549.84 *i 

S ,9604 

5.9602 

46550.17 i 

'5035 l 

5000 

i 

5003 

5002 

I 7.0 

40217.27 

1 6.9641 

40208.57 I 

6.9659 

6.9653 

40208.84 ! 

5026 i 

sooo 

i 

i 

5002 

5001 

} 8.0 

3S381.2? 

1 7.9686 

35373.64 

7,9701 

7.9693 

35373.87 i 

5020 S 

5000 

1 

S002 

5000 

2 9.0 

31578.99 

1 8.9721 

31572,17 1 

8.9734 

8.9726 

31572.37 ! 

5016 { 

5000 

\ 

5002 

5000 

1 10.0 

28514.93 

1 9.9750 

28508.77 

9.9761 

9.9752 

28508.96 1 

5013 1 

5000 


5002 

5000 

2 12.0 

23888.82 

1 11.9793 

23883.66 

11.9801 

11 .9793 

23883.82 i 

5010 J 

5000 


5001 

5000 

! 14.0 

20568.67 

1 13.9824 

20564.23 ! 

I 13.9830 

13.9823 

20564,37 \ 

5007 ! 

5000 


5001 

5000 

2 16.0 

18074.63 

1 15.9847 

18070.73 t 

15.9852 

15.9845 

18070.86 } 

SO 06 ! 

5000 


5001 

5000 

2 18.0 

16135.69 

1 17.9866 

16132.21 ; 

I 17.9869 

! 17.9863 

16132,32 ! 

5005 ! 

5000 


5001 

50 00 

2 20.0 

14587.44 

1 19.9880 

14584.29 ! 

19.9883 

19.9878 

14584.39 ! 

5004 S 

5000 


5001 

5000 

2 25.0 

11816.33 

1 24.9908 

11813.79 ! 

1 24.9909 

! 24.9904 

11813.87 ! 

5003 : 

5000 


5001 

5000 

2 30.0 

9992,57 

\ 29.9926 

9990.41 J 

1 29.9926 

I 29.9923 

9990.49 ! 

5002 ; 

5000 


5001 

5000 

2 ' 35.0 

8713.37 

1 34.9940 

8711.49 ! 

! 34.9939 

! 34.9936 

8711.55 1 

5002 } 

5000 


5001 

5000 

2 40.0 

7776.72 

1 39.9950 

7775.05 j 

: 39.9949 

39.9947 

7775.10 ! 

5002 ! 

5000 

\ 

5001 

5000 

1 45.0 

7070.29 

! 44.9958 

7068.76 

1 44.9957 

! 44.9955 

7063,81 I 

5001 | 

5000 


5001 

5000 

1 50,0 

6526.90 

i 49.9965 

6525,50 J 

! 49.9964 

: 49,9962 

6525.54 ! 

5001 i 

5000 


5001 

5000 

2 60.0 

5774.07 

1 59.9977 

5772.03 

! 59.9975 

1 59.9974 

5772.87 ! 

5001 { 

5000 


5001 

5000 

2 70.0 

5321.71 

I 69.9985 

5320.57 J 

! 69.9985 

! 69.9984 

5320.61 ! 

S001 i 

5000 


500 1 

5000 

2 ' 80.0 

5078.12 

1 79.9993 

5077.03 

! 79,9993 

i 79.9992 

5077.07 i 

5001 1 

5000 


5001 

5000 

1 90,0 

i 

% ^ ^ 

5001.04 

: 90.0000 

i 

4999.96 ! 

1 90.0000 
1 

I 90.0000 
1 

5000.00 : 
: 

5001 1 

> 
1 

5000 


5001 

5000 

***** INDICATES INPUT 

VALUES HAVE 

EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 






COMPARISON OF GRADIENT (GMD>, WHITE SANDS (CYBER) AND SPHERICAL SLAB < VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR ..THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM DAY AFTERNOON SEA LEVEL INDEX OF REFRACTION: .0002866 SCALE HEIGHT: 7406.42 METERS 


1 INPUT 

DATA 

1 GMD GRAD 

REF SOU 

CYBER 

SPHERICAL SLAB S 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD 

! E MEAS 

R MEAS 

l E COR 

R COR 

E COR 

E COR 

R COR t 

NO CQRR 



GMD H 



CYB H 


UAR H 

1 DEG ' * 

METER 

; DEG 

METER 

l DEG 

DEG 

METER \ 

METER 


t 

METER 



METER 


METER 

l — 

! 0.0 

401494.96 

»- 

1 -.3745 

401409.34 

t 

-.4606 

-.2665 

i 

401423.45 \ 

12624 


i 

10000 



9402 


10756 

i . 5 ■ • 

335350.98 

1 .2043 

335284.04 

1 .1806 

.2547 

' 335288.67 \ 

11732 


J 

10000 



9865 


10295 

I i.O 

282648.81 

I .7598 

282594.76 

.7589 

.7938 

282598.79 \ 

11186 



10000 



9998 


10167 

i 1.5 

240803.34 

i 1.3004 

240758.58 

l 1'. 3083 

1.3253 

240762.37 \ 

10840 


t 

10000 



10036 


10105 

; 2.o 

207587.03 

t 1.8308 

207549.18 

1 .8420 

1.8490 

207552.68 S 

10614 



10000 



10043 


10066 

J 2.5 

181100.35 

1 2.3542 

181067.75 

2.3662 

2.3671 

181070.89 * 

10462 



10000 



10040 


10041 

: 3.0 

159801.25 

! 2.8725 

159772.75 

2.8844 

2.8816 

159775.51 ! 

10357 



10000 



10035 


10026 

I 3.5 

142488.40 

* 3.3870 

142463.16 

3.3985 

3.3942 

142465.56 ! 

10282 



10000 



10030 


10018 

! 4.0 

128250.19 

i 3.8989 

128227.58 . 

3.9097 

3.9061 

128229.67 1 

10227 



10000 



10026 


10016 

J 4.5 

116401.59 

1 4.4086 

116381.15 

4.4187 

4.4179- 

116382.98 1 

10187 



10000 



10022 


10019 

\ 5.0 

106428.91 

! 4.9167 

106410.27 

4.9262 

4.9264 

106411.89 i 

101S6 



10000 



10019 


10018 

J 6.0 

90658.14 

» 5.9295 

90642.33 

5.9378 

5.9362 

90643.62 i 

10113 



10000 



10015 


iOOil 

! 7,0 

78821.35 

J 6.9390 

78807.63 

6.9463 

6.9440 

78808.73 ! 

10085 



10000 



10012 


10007 

: 8.o 

69653,44 

1 7.9464 

69641.34 ! 

l 7.9528 ! 

7.9503 

69642.30 i 

10066 



10000 



10 010 


10005 

: ?.o 

62365.49 

I 8.9522 

62354.67 J 

! 8.9580 { 

8.9553 

62355.54 i 

10053 



10000 



10008 


1 0004 

! 10.0 

56445.39 

» 9.9569 

56435.60 

! 9.9621 

9.9595 

56436.41 ! 

10043 



10000 



10007 


10003 

} 12.0 

47434.68 

J 11.9642 

47426.47 1 

I- 11.9685 ! 

11 .9660 

47427.16 i 

10031 



10000 



10005 


10002 

J 14.0 

40920.31 

! 13.9694 

40913.23 ! 

\ 13.9731 I 

13.9708 * 

40913.84 \ 

10023 



10000 



1 0 0 04 


10001 

! 16.0 

36003.90 

! 15.9734 

35997.67 ! 

1 15.9766 ! 

15.974S 

35998.21 \ 

10018 



10000 



10004 


10001 

J 18.0 

32169.66 

1 17.9765 

32164.10 

1 17.9793 1 

17.9775 

32164.59 \ 

10014 



10000 



10003 


10001 

: 2 o,o 

29101. 08 

! 19.9791 

29096.05 I 

19.9815 J 

19.9798 

29096.50 \ 

10012 



10000 



10003 


10000 

! 25.0 

23595.72 

J 24.9837 

23591.64 ! 

! 24.9855 *i 

24.9842 

*23592,00 ! 

10008 



10000 


l 

10002 


10000 

30.0 

19964.53 

J 29.9869 

19961.08 ! 

29.9883 J 

29.9872 

19961.38 \ 

10006 



10000 



10002 


10 0 00 

! 35 . 0 

17414.33 

1 34.9892 

17411.32 1 

I 34.9904 i 

34.9894 

17411.59 1 

10004 



10000 



10002 


10000 

! 40.0 

15545,48 

{ 39.9911 

15542,80 J 

39,9920 ! 

39.9912 

15543.04 : 

10004 



10000 



10002 


i 0 0 0 0 

• 45.0 

14135.31 

\ 44.9925 

14132,86 J 

! 44.9932 ! 

44.9926 

14133.08 I 

10003 



10000 



10002 


I 10000 

l 50.0 

13050 . 32 

! 49.9937 

13048.06 } 

49.9943 S 

49.9938 

13048.26 ! 

10003 



10000 



10002 


10 0 00 

: 60.0' 

11546.42 

! 59.9957 

11544.42 ! 

i 59.9961 I 

59.9957 

11544.60 ! 

10002 



10000 


| 

10 002 


; 10000 

: 7o.o 

10642.65 

! 69.9973 

10640.81 i 

69.9975 1 

69.9973 

10640.97 J 

10002 



10000 



10002 


10 0 0 0 

.* &0.0 

10155.75 

1 79.9987 

10153.99 ! 

1 79.9988 S 

79.9987 

10154.15 ; 

10002 



10000 



100 02 


10 0 0 0 

I 90.0 

10001 . 68 

i 90.0000 

t 

i 

9999.95 \ 
1 

90.0000 : 

: i 

90.0000 

10000.11 1 
1 
I 

10002 



10000 

i 

t 

10002 


10000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIHITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYDER > AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE VS-FM-60 


WEATHER DATA FOR TYPICAL WARM DAY AFTERNOON SEA LEVEL INDEX OF REFRACTION l .0002866 SCALE HEIGHT! 7406.42 METERS 


INPUT 

DATA ! 

GMD GRAD 

REF SOL i 

CYDER i 

SPHERICAL SLAB 1 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD i 

E MEAS 

R MEAS i 

E COR 

R COR 1 

E COR 1 

E COR 

R COR i 

NO CORR 

GMD H 


CYB H 

VAR H t 

DEC 

METER i 

DEG 

METER ! 

DEG ! 

DEG 

METER i 

METER 

METER 

i 

METER 

METER 

.... ... ... ... .. _m i_ 

0,0 

• 

• 

559774.64 S 

-.4632 

l 

559678.32 1 

* 

i 

-.6422 1 

-.3164 

i 

» 

559700.37 i 

24517 

20000 


18263 

j 

21431 i 

.5 

492092.93 l 

.1228 

492016.02 t 

.0657 1 

.2104 

492024.78 i 

23235 

20000 

» 

19518 

20751 

i.O 

435052.75 i 

.6847 

434989.37 t 

.6739 i 

.7485 

434996.57 ! 

22391 

20000 


19923 

20483 ! 

1.5 

386743.62 i 

1.2313 

386690.16 I 

1.2402 S 

1.2799 

386696.53 i 

21812 

20000 

\ 

20064 

20328 

2.0 

345759.01 t 

1.7673 

345713.07 l 

1.7849 i 

1.8048 

345718.78 ! 

21403 

20000 

\ 

20110 

20226 ! 

2.5 

310935.97 i 

2.2958 * 

310895.88 * 

2.3170 1 

2.3246 

310900.95 \ 

21107 

20000 


20118 

20156 

3.0 

281277.07 l 

2,8188 

281241.64 S 

2.8411 i 

2.B412 

281246.10 S 

20889 

20000 


20112 

20110 

3,5 

255927.68 ! 

3.3376 

255896.02 S 

3.3599*1 

3.3562 

255899.94 5 

20725 

20000 


20102 

20083 

4.0 

234164. S6 S 

3.8532 

234135.99 I 

3.8748 S 

3.8709 

234139.45 l 

20600 

20000 


20091 

20072 

4.5 

215383.80 ! 

4.3663 

215357.82 i 

* 4.3870 i 

4.3853 

215360.89 I 

20503 

20000 


20080 

20073 

5.0 

199086.39 J 

4.8774 

199062.59 ! 

4.8971 i 

4.8965 

199065.33 ! 

20426 

20000 


20071 

20066 

6.0 

172377.72 S 

5.8951 

172357.38 t 

5.9129 i 

5.9097 

172359.63 i 

20315 

20000 


£0056 

20044 

7.0 

151577.18 i 

6.9087 

151559.46 1 

6.9246 ! 

6.9203 

151561 . 4 0 S 

20242 

20000 


20044 

20031 

8.0 

*135032.39 1 

7.9193 

135016.71 t 

7.9337 i 

7.9289 

135018.44 i 

20190 

20000 


20036 

20023 

9.0 

121620.85 l 

8.9278 

121606.7 8 1 

8.9408 i 

8.9360 

121608.36 ! 

20153 

20000 


20030 

20017 

10.0 

110565.41 ! 

9.9347 ' 

110552.67* 1 

9.9466 1 

9.9419 

110554.13 I 

20126 

20000 


20025 

20014 ! 

12.0 

93480.82 i 

11.9454 

93470.10 I 

11.9555 I 

11.9511 

93471.37 ! 

20089 

20000 


20018 

20009 I 

14.0 

80951.90 ! 

13.9533 

80942.64 l 

13.9619 1 

1 13.9580 

80943.76 l 

20066 

20000 


20014 

20007 

16.0 

71408.16 1 

15.9593 

71400.01 : 

15.9668 i 

15.9632 

71401.01 i 

20051 

20000 


20011 

20005 ! 

18.0 

63917.53 i 

17.9640 

63910.24 { 

17.9707 i 

! 17.9674 

63911.14 S 

20040 

* 20000 


20009 

20004 

20.0 

57895.56 i 

19.9678 

57888.97 J 

19.9738 1 

19.9708 

57889.79 i 

20033 

20000 


20008 

20003 

25. 0 

47037.48 ! 

24.9749 

47032.13 i 

24.9795 ! 

! 24.9771 

47032.80 \ 

20021 

20000 


20006 

20002 

30.0 

39842.82 l 

29.9797 

39838.29 ! 

29.9834 ! 

1 29.9815 

39838.86 ! 

20014 

20000 


20005 

20001 

35.0 

34776.77 ' 

34.9833 

34772.82 ! 

34.9863 I 

S 34,9847 

34773.32 *, 

20011 

20000 


20004 

20001 

40.0 

31058.15 ! 

39.9861 

31054.62 i 

39.9886 ! 

I 39.9872 

31055.07 1 

20008 

20000 


20003 

20001 

4S . 0 

28249.11 ! 

44.9883 

28245.91 1 

44.9904 

! 44.9893 

28246.31 ; 

20006 

20000 


20 0 03 

20001 

50.0 

26086.14 1 

49.9902 

26083.18 l 

49.9920 \ 

1 49.9910 

26083.56 5 

20005 

20000 


20003 

2000 1 

60.0 

23086.15 ! 

59.9933 

23083.53 l 

59.9945 

1 59.9938 

23083.87 ! 

20004 

20000 


20002 

20000 

70.0 

21282.13 i 

69.9958 

21279.71 i 

69.9965 ! 

! 69.9961 

21280.02 ! 

20003 

20000 


20002 

20000 

I 80.0 

20309.96 i 

79.9980 

20307.66 t 

79.9983 

! 79.9901 

20307.95 ! 

20002 

20000 


20 0 02 

20000 

90.0 

20002.26 i 
• 
• 

90.0000 

19999.99 \ 
\ 

90.0000 1 

1 90.0000 
1 ’ 

20000.28 i 
1 
1 

20002 

20000 


20002 

20000 


***** INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 



COMPARISON OF GRADIENT (GMD) , WHITE SANDS (CYDER) AND SPHERICAL SLAB ( VARIAN) CORRECTION METHODS 
BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAKE MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 75-FM-60 


WEATHER DATA FOR TYPICAL WARM DAY AFTERNOON SEA LEVEL INDEX OF REFRACTION; 


.0002866 


SCALE HEIGHT: 7406.42 METERS 


INPUT DATA 


\ GMD GRAD REF SOL 


l CYBER 


SPHERICAL SLAB 


! ALTITUDE COMPUTED BY EACH SOLUTION METHOD 


1 E MEAS 
i DEG • 

R MEAS \ 

METER i 

E COR 
DEG 

R COR 
METER 

i 0.0 

S 

864442.00 t 

-.5550 

864342,12 

1 .5 

795508.26 *1 

.0428 

795428; 57 

I 1.0 

734817. SS \ 

• .6140 

734751.86 

l 1.5 

680729.69 1 

1.1679 

680674.22 

l 2.0 

632226.99 i 

1.7100 

632179.26 

! 2.5 

588584.99 J 

2.2436 

588543.28 

i 3.0 

549234,58 ‘ i 

2.7709 

549197.65 

\ 3.5 

513701 , 28 i 

3.2935 

513668.23 

\ 4.0 

481575.11 ! 

3 .8123 

481545.26 

1 4.5 

4S2484.00 ! 

4.3283 

452456.81 

ta ! 5.0 

426099.43' ! 

4.8420 

426074.49 

' ! 6>0 

380334.08 i 

5.8640 

380312.74 

^ ! 7.0 

342311.48 i 

6,8810 

342292. 8S 

i 8.0 

310462.06 1 

7.8944 

310445.56 

! 9.0 

283556.09 l 

8.9053 

283541.28 

! 10.0 

260629.66 J 

9.9142 

260616.23 

l 12.0 

223864.19 i 

11.9280 

223852.88 

l 14.0 

195903.55 ! 

13.9382 

195893.78 

i 16.0 

174066.69 i 

15.9460 

174058.08 

! 18.0 

156620.29 J 

17.9522 

156612.60 

} 20.0 

142411.16 J 

19.9572 

142404.20 

! 25.0 

116411.37 S 

24.9665. 

116405.72 

! 30.0 

98941.59 i 

29.9729 

98936.81 

! 35.0 

86541.28 \ 

34.9777 

86537.11 

I 40.0 

77392.86 ! 

39,9814 

77389.14 

! 45.0 

70458.83 ! 

44.9843 

70455.44 

i 50.0 

65107.07 S 

49,9869 

65103.94 

i 60.0 

57667.79 1 

S9.9910 

57665.02 

i 70.0 

53185.73 l 

69.9943 

53183,18 

1 80.0 

50768.00 l 

79.9972 

50765.57 

! 90.0 

50002.40 l 

90.0000 

50000.00 


E COR 
DEG 


E COR 
DEG 


R COR 1 
METER I 


NO CORR : 
METER ! 


GMD H 
METER 


CYB H 
METER 


VAR H I 
METER I 


-.9918 

! -.4010 

864367.16 t 

58313 \ 

50000 i 

43469 : 

5230 5 

-.1146 

t - .1227 

795436.80 ! 

56303 S 

50000 ! 

47839 J 

51100 

* .5554 

1' .6751 

734758.63 1 

54916 ! 

50000 i 

49261 ! 

50777 

1.1522 

i 1.2171 

680680.23 1 

53918 ! 

50000 1 

49821 S 

50579 

1.7148 

1 1.7497 

632184.64 S 

53177 ! 

50000 S 

50057 t 

50433 

2.258S 

1 2.2754 

588548.05 » 

52614 ! 

50000 \ 

50156 : 

50322 

2.7908 

! 2.7967 

549201.83 i 

52179 ! 

50000 i 

50192 . 

50243 

3.3157 

i 3.3157 

513671.88 ! 

51837 S 

50000 : 

50200 \ 

50197 

3.8353 

* -3.8343 

481548.45 i 

51565 i 

50000 S 

50195 i 

50183 

4.3513 

\ 4.3532 

4S2459.63 I 

51345 l 

50000 J 

50183 { 

50195 

4.8645 

1 4.8663 

426077.00 ! 

51165 J 

50000 \ 

50169 i 

50178 

5.8851 

i 5.8832 

380314.78 ! 

50892 i 

50000 ; 

50140 i 

50125 

6.9004 

i 6.8969 

342294.61 i 

50702 ! 

50000 J 

50116 S 

50092 

7.9122 

1 7.9080 

310447.14 S 

50564 S 

50000 t 

50096 J 

50071 

8.9216 

! 8.9171 

283542.73 i 

50461 S 

50000 S 

50031 ; 

50057 

9.9292 

i 9.9247 

260617.59 i 

50384 ! 

50000 i 

50069 : 

50046 

11.9409 

11.9365 

223854.07 J 

50274 : 

50000 { 

50050 ; 

50031 

13.9494 

{ 13.9454 

195894.83 l 

50204 i 

50000 t 

50038 • 

50022 

15.9559 

J 15.9522 

174059.03 ! 

5-0157 ! 

50000 1 

50029 J 

50017 

17.9610 

\ 17.9577 

1S6613.45 I 

50124 : 

50000 J 

50024 1 

50013 

19.9651 

l 19.9621 

142404.98 J 

50101 1 

50000 { 

50020 : 

50010 

24.9727 

i 24.9703 

116406.36 \ 

50063 S 

50000 ! 

50014 ! 

50007 

29.9779 

i 29.9759 

98937.36 ! 

50042 i 

50000 S 

50009 S 

50004 

34.9818 

\ 34.9801 

86537.59 i 

50029 : 

50000 1 

50007 ! 

50002 

39.9848 

! 39.9834 

77389.57 ! 

50021 ! 

50000 : 

50005 : 

50001 

44.9872 

! 44.9860 

70455.83 1 

50015 ; 

50000 J 

50004 i 

50001 

49.9893 

‘ 49.9883 

65104.30 1 

50011 1 

50000 { 

50003 J 

50001 

59.9926 

! 59.9919 

57665.34 ! 

50006 S 

50000 ! 

50003 ; 

50000 

69.9954 

1 69.9949 

53183.48 1 

50004 l 

50000 ! 

50003 ! 

5000 0 

79.9977 

! 79.9975 

50765.85 J 

50003 : 

50000 ,* 

50002 i 

50000 

90.0000 

I 90.0000 
\ 

50000.28 : 
1 
1 

50002 ! 

1 

I 

50000 ! 

• 

• 

50002 ! 

t 

i 

50000 


1***% INDICATES INPUT VALUES HAVE EXCEEDED THE COMPUTATIONAL LIMITS OF THE SOLUTION METHOD 
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COMPARISON OF GRADIENT (GMD), WHITE SANDS (CYDER) AND SPHERICAL SLAB (VARIAN) CORRECTION METHODS 
' BASED ON RESULTS OF GRADIENT REFRACTION SOLUTION FOR THE SAME MEASURED EL ANGLES AS USED IN JSC INTERNAL NOTE 7S-FM-60 


WEATHER DATA FOR TYPICAL WARM DAY AFTERNOON SEA LEVEL INDEX OF REFRACTION! .0002866 SCALE HEIGHT! 7406.42 METERS 


— 
1 INPUT 

DATA l 

GMD GRAD 

REF SOL t 

CYBER * 

SPHERICAL SLAB I 

ALTITUDE 

COMPUTED BY 

EACH SOLUTION METHOD ! 

E MEAS 

‘ R HEAS i 

E COR* 

R COR ! 

E COR 1 

E COR 

R COR : 

NO CORR 1 


GMD H 


CYB H ! 

VAR H 1 

DEG 

METER ! 

DEG 

METER I 

DEG i 

DEG 

METER ! 

METER I 


METER 


METER 

METER l 

! 0.0 

1 

1203073.35 ! 

“*T“ 

-.6031 

* 

1202972.38 \ 

1 

-1.3803 S 

-.4511 

S 

1202998.47 1 

: 

112472 i 


100000 


83932 

— — ” ““-"L. ' ' “ | 

l 

103145 ! 

\ .5 

1133543.53 1 

.0035 

1133463.21 S 

-.2690 1 

.0695 

1133470.67 t 

109677 S 


100000 


94703 

101285 

S 1.0 

1071088.79 1 

.5810 

1071022.72 * 

.4672 i 

.6331 

1071029.01 1 

107718 ! 


100000 ‘ 


97916 

100959 : 

: i .s 

1014120.78 i 

1 . 1397 

1014065.07 \ 

1.0926 : 

1.1831 

1014070.79 S 

106203 : 


100000 


99188 

100757 

; 2.o 

961708.66 i 

1.6855 

961660.75 1 

1.6703 i 

1 .7215 

961665.96 i 

105199 I 


100000 


99756 

100595 ; 

: 2.5 

913254.38 

2.2220 

913212.54 S 

2.2233 ! 

2.2514 

913217.19 ! 

104364 i 


100000 

1 

100027 

100461 

! 3.0 

868304.95 ! 

2.7517 

868267.94 \ 

2.7617 ! 

2.7759 

868272 . 04 1 

103705 i 


100000 


100155 

100360 

! 3;5 

826526.62 

3.2761 

'826493.51 * 

3.2908 1 

3.2976 

826497.09 ! 

103177 


100000 

1 

100213 

100302 

{ 4.0 

787649.73 I 

3.7966 

787619.82 i 

3.8137 S 

3.8184 

787622.9 7 ! 

102750 i 


100000 


100235 

100293 

! 4.5 

751438.57 

4.3139 

751411.34 S 

4.3321* 

4.3394 

751414 . 12 i 

102400 


100000 

{ 

100239 

100329 

1 5.0 

717677.32 ! 

4.8287 

717652.35 ! 

4.8473 J 

4.8539 

717654.82 I 

102109 S 


100000 


100233 

100310 

i 6.0 

656782.0 7 

5.8525 

656760.71 ! 

5.8707 

5.8727 

656762.72 ! 

101658 


100000 


100207 

100225 

\ 7.0 

603670.82 1 

6.8709 

603652.18 S 

6.8880 l 

6.8377 

603653.92 i 

101332 


100000 


100179 

100172 

J 8.0 

557223.77 ! 

! 7.8854 

557207.26 ! 

7.9014' J 

7.8998 

557208.82 ! 

101090 


100000 


100155 

100137 

: 9.0 

516458.97 1 

8.8971 

516444.16 I 

8.9119 ! 

8.9098 

516445.60 i 

100903 


100000 


100131 

100110 

I 10 .0 

480S79 , 16 1 

! 9.9068 ‘ 

480565.73 ! 

9.9205 ! 

9.9181 
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